Acknowledgments |
|
xv | |
About The Authors |
|
xvii | |
About The Technical Editor |
|
xix | |
Foreword |
|
xxi | |
Preface |
|
xxv | |
|
Chapter 1 Logs, Trees, Forest: The Big Picture |
|
|
1 | (28) |
|
|
1 | (1) |
|
|
2 | (13) |
|
|
2 | (2) |
|
How is Log Data Transmitted and Collected? |
|
|
4 | (2) |
|
|
6 | (1) |
|
|
7 | (8) |
|
|
15 | (1) |
|
|
16 | (1) |
|
|
17 | (6) |
|
|
17 | (1) |
|
|
18 | (3) |
|
|
21 | (1) |
|
|
21 | (1) |
|
Boring Audit, Fun Discovery |
|
|
22 | (1) |
|
People, Process, Technology |
|
|
23 | (1) |
|
Security Information and Event Management (SIEM) |
|
|
24 | (3) |
|
|
27 | (2) |
|
|
29 | (22) |
|
|
29 | (3) |
|
|
29 | (3) |
|
|
32 | (14) |
|
|
34 | (6) |
|
|
40 | (4) |
|
|
44 | (2) |
|
|
46 | (2) |
|
|
47 | (1) |
|
|
48 | (3) |
|
Chapter 3 Log Data Sources |
|
|
51 | (20) |
|
|
51 | (1) |
|
|
51 | (12) |
|
|
52 | (6) |
|
|
58 | (4) |
|
|
62 | (1) |
|
Log Source Classification |
|
|
63 | (7) |
|
Security-Related Host Logs |
|
|
64 | (4) |
|
Security-Related Network Logs |
|
|
68 | (1) |
|
|
68 | (2) |
|
|
70 | (1) |
|
Chapter 4 Log Storage Technologies |
|
|
71 | (22) |
|
|
71 | (1) |
|
|
71 | (2) |
|
|
73 | (5) |
|
|
73 | (3) |
|
|
76 | (1) |
|
|
76 | (2) |
|
Database Storage of Log Data |
|
|
78 | (3) |
|
|
78 | (1) |
|
|
78 | (1) |
|
Defining Database Storage Goals |
|
|
79 | (2) |
|
|
81 | (1) |
|
|
82 | (1) |
|
|
82 | (1) |
|
|
82 | (7) |
|
Getting Started with Amazon Elastic MapReduce |
|
|
83 | (1) |
|
|
83 | (1) |
|
Uploading Logs to Amazon Simple Storage Services (S3) |
|
|
84 | (2) |
|
Create a Pig Script to Analyze an Apache Access Log |
|
|
86 | (1) |
|
Processing Log Data in Amazon Elastic MapReduce (EMR) |
|
|
87 | (2) |
|
Log Data Retrieval and Archiving |
|
|
89 | (1) |
|
|
90 | (1) |
|
|
90 | (1) |
|
|
90 | (1) |
|
|
90 | (3) |
|
Chapter 5 syslog-ng Case Study |
|
|
93 | (10) |
|
|
93 | (1) |
|
|
93 | (1) |
|
What Is syslog-ngsyslog-ng? |
|
|
94 | (1) |
|
|
95 | (4) |
|
|
96 | (3) |
|
Troubleshooting syslog-ng |
|
|
99 | (2) |
|
|
101 | (2) |
|
|
103 | (12) |
|
|
103 | (2) |
|
Complete Stealthy Log Setup |
|
|
105 | (5) |
|
|
105 | (1) |
|
|
106 | (1) |
|
|
106 | (1) |
|
|
107 | (2) |
|
"Fake" Server or Honeypot |
|
|
109 | (1) |
|
|
110 | (3) |
|
Honeynet's Shell Covert Keystroke Logger |
|
|
111 | (1) |
|
Honeynet's Sebek2 Case Study |
|
|
112 | (1) |
|
Covert Channels for Logging Brief |
|
|
113 | (1) |
|
|
114 | (1) |
|
Chapter 7 Analysis Goals, Planning, and Preparation: What Are We Looking For? |
|
|
115 | (12) |
|
|
115 | (1) |
|
|
115 | (2) |
|
|
115 | (2) |
|
Future Bad Things, Never Before Seen Things, and All But the Known Good Things |
|
|
117 | (1) |
|
|
117 | (5) |
|
|
117 | (1) |
|
|
118 | (1) |
|
|
119 | (1) |
|
|
119 | (1) |
|
|
120 | (1) |
|
|
120 | (1) |
|
|
121 | (1) |
|
|
122 | (3) |
|
|
122 | (1) |
|
|
122 | (1) |
|
|
122 | (3) |
|
|
125 | (2) |
|
Chapter 8 Simple Analysis Techniques |
|
|
127 | (18) |
|
|
127 | (1) |
|
Line by Line: Road to Despair |
|
|
127 | (2) |
|
|
129 | (5) |
|
|
129 | (1) |
|
|
130 | (1) |
|
|
131 | (3) |
|
Limitations of Manual Log Review |
|
|
134 | (1) |
|
Responding to the Results of Analysis |
|
|
135 | (5) |
|
|
135 | (2) |
|
Acting on Summaries of Non-Critical Logs |
|
|
137 | (1) |
|
Developing an Action Plan |
|
|
138 | (2) |
|
|
140 | (1) |
|
|
140 | (2) |
|
Incident Response Scenario |
|
|
140 | (1) |
|
|
141 | (1) |
|
|
142 | (3) |
|
Chapter 9 Filtering, Normalization, and Correlation |
|
|
145 | (36) |
|
|
145 | (2) |
|
|
147 | (1) |
|
|
147 | (1) |
|
|
148 | (6) |
|
|
150 | (1) |
|
|
150 | (1) |
|
|
150 | (1) |
|
Generic Cisco IOS Messages |
|
|
151 | (1) |
|
Regular Expression Performance Concerns |
|
|
152 | (2) |
|
|
154 | (24) |
|
|
155 | (2) |
|
|
157 | (4) |
|
Using Data in Your Environment |
|
|
161 | (1) |
|
Simple Event Correlator (SEC) |
|
|
161 | (2) |
|
|
163 | (6) |
|
Building Your Own Rules Engine |
|
|
169 | (9) |
|
Common Patterns to Look For |
|
|
178 | (1) |
|
|
178 | (2) |
|
|
180 | (1) |
|
Chapter 10 Statistical Analysis |
|
|
181 | (12) |
|
|
181 | (1) |
|
|
181 | (1) |
|
|
182 | (5) |
|
|
186 | (1) |
|
|
186 | (1) |
|
|
187 | (1) |
|
|
187 | (3) |
|
|
188 | (1) |
|
Applying the k-NN Algorithm to Logs |
|
|
188 | (2) |
|
Combining Statistical Analysis with Rules-Based Correlation |
|
|
190 | (1) |
|
|
191 | (2) |
|
Chapter 11 Log Data Mining |
|
|
193 | (14) |
|
|
193 | (1) |
|
|
194 | (4) |
|
|
198 | (2) |
|
|
200 | (1) |
|
|
201 | (2) |
|
|
203 | (2) |
|
|
205 | (2) |
|
Chapter 12 Reporting and Summarization |
|
|
207 | (12) |
|
|
207 | (1) |
|
Defining the Best Reports |
|
|
208 | (3) |
|
Authentication and Authorization Reports |
|
|
208 | (3) |
|
|
211 | (2) |
|
|
211 | (1) |
|
|
212 | (1) |
|
Who Can Use These Reports |
|
|
213 | (1) |
|
|
213 | (2) |
|
|
213 | (1) |
|
|
213 | (1) |
|
Who Can Use These Reports |
|
|
214 | (1) |
|
|
215 | (1) |
|
|
215 | (1) |
|
|
215 | (1) |
|
Who Can Use These Reports |
|
|
216 | (1) |
|
Critical Errors and Failures Reports |
|
|
216 | (1) |
|
|
216 | (1) |
|
|
216 | (1) |
|
Who Can Use These Reports |
|
|
217 | (1) |
|
|
217 | (2) |
|
Chapter 13 Visualizing Log Data |
|
|
219 | (12) |
|
|
219 | (1) |
|
|
219 | (1) |
|
|
220 | (1) |
|
|
221 | (1) |
|
|
222 | (5) |
|
Traditional Log Data Graphing |
|
|
227 | (2) |
|
|
229 | (2) |
|
Chapter 14 Logging Laws and Logging Mistakes |
|
|
231 | (12) |
|
|
231 | (1) |
|
|
231 | (3) |
|
Law 1---Law of Collection |
|
|
232 | (1) |
|
|
232 | (1) |
|
Law 3---Law of Monitoring |
|
|
233 | (1) |
|
Law 3---Law of Availability |
|
|
233 | (1) |
|
|
233 | (1) |
|
Law 5---Law of Constant Changes |
|
|
234 | (1) |
|
|
234 | (7) |
|
|
235 | (1) |
|
|
236 | (1) |
|
Storing for Too Short a Time |
|
|
237 | (2) |
|
Prioritizing Before Collection |
|
|
239 | (1) |
|
Ignoring Application Logs |
|
|
240 | (1) |
|
Only Looking for Known Bad Entries |
|
|
241 | (1) |
|
|
241 | (2) |
|
Chapter 15 Tools for Log Analysis and Collection |
|
|
243 | (24) |
|
|
243 | (1) |
|
|
243 | (4) |
|
|
244 | (1) |
|
|
245 | (1) |
|
|
246 | (1) |
|
Questions for You, Your Organization, and Vendors |
|
|
246 | (1) |
|
Basic Tools for Log Analysis |
|
|
247 | (7) |
|
|
247 | (2) |
|
|
249 | (2) |
|
|
251 | (1) |
|
Other Basic Tools to Consider |
|
|
252 | (2) |
|
The Role of the Basic Tools in Log Analysis |
|
|
254 | (1) |
|
Utilities for Centralizing Log Information |
|
|
254 | (3) |
|
|
254 | (2) |
|
|
256 | (1) |
|
|
256 | (1) |
|
Log Analysis Tools---Beyond the Basics |
|
|
257 | (5) |
|
|
257 | (4) |
|
|
261 | (1) |
|
Other Analysis Tools to Consider |
|
|
261 | (1) |
|
|
262 | (3) |
|
|
263 | (1) |
|
|
264 | (1) |
|
|
264 | (1) |
|
|
265 | (1) |
|
|
265 | (2) |
|
Chapter 16 Log Management Procedures: Log Review, Response, and Escalation |
|
|
267 | (38) |
|
|
267 | (1) |
|
Assumptions, Requirements, and Precautions |
|
|
268 | (1) |
|
|
269 | (1) |
|
|
269 | (1) |
|
Common Roles and Responsibilities |
|
|
269 | (1) |
|
|
270 | (7) |
|
|
271 | (4) |
|
Other Requirements Related to Logging |
|
|
275 | (2) |
|
|
277 | (1) |
|
Review, Response, and Escalation Procedures and Workflows |
|
|
278 | (15) |
|
Periodic Log Review Practices and Patterns |
|
|
279 | (4) |
|
Building an Initial Baseline Using a Log Management Tool |
|
|
283 | (2) |
|
Building an Initial Baseline Manually |
|
|
285 | (1) |
|
Main Workflow: Daily Log Review |
|
|
286 | (3) |
|
Exception Investigation and Analysis |
|
|
289 | (2) |
|
Incident Response and Escalation |
|
|
291 | (2) |
|
|
293 | (3) |
|
|
294 | (1) |
|
|
294 | (1) |
|
Proof of Exception Handling |
|
|
294 | (2) |
|
Logbook---Evidence of Exception of Investigations |
|
|
296 | (3) |
|
Recommended Logbook Format |
|
|
296 | (1) |
|
|
297 | (2) |
|
PCI Compliance Evidence Package |
|
|
299 | (1) |
|
|
300 | (1) |
|
Periodic Operational Tasks |
|
|
300 | (3) |
|
|
300 | (1) |
|
|
300 | (1) |
|
|
301 | (1) |
|
|
302 | (1) |
|
|
303 | (1) |
|
|
303 | (1) |
|
|
303 | (2) |
|
Chapter 17 Attacks Against Logging Systems |
|
|
305 | (24) |
|
|
305 | (1) |
|
|
305 | (22) |
|
|
306 | (1) |
|
Attacks on Confidentiality |
|
|
307 | (6) |
|
|
313 | (5) |
|
|
318 | (9) |
|
|
327 | (2) |
|
Chapter 18 Logging for Programmers |
|
|
329 | (14) |
|
|
329 | (1) |
|
Roles and Responsibilities |
|
|
329 | (2) |
|
|
331 | (9) |
|
|
332 | (1) |
|
Logging APIs for Programmers |
|
|
333 | (2) |
|
|
335 | (1) |
|
|
336 | (1) |
|
|
337 | (3) |
|
|
340 | (1) |
|
Performance Considerations |
|
|
341 | (1) |
|
|
342 | (1) |
|
Chapter 19 Logs and Compliance |
|
|
343 | (24) |
|
|
343 | (1) |
|
|
344 | (6) |
|
|
345 | (5) |
|
|
350 | (3) |
|
|
353 | (7) |
|
|
360 | (6) |
|
NIST 800-53 Logging Guidance |
|
|
361 | (5) |
|
|
366 | (1) |
|
Chapter 20 Planning Your Own Log Analysis System |
|
|
367 | (14) |
|
|
367 | (1) |
|
|
367 | (4) |
|
Roles and Responsibilities |
|
|
368 | (1) |
|
|
368 | (2) |
|
|
370 | (1) |
|
Selecting Systems and Devices for Logging |
|
|
371 | (1) |
|
|
371 | (3) |
|
|
371 | (1) |
|
|
372 | (2) |
|
|
374 | (2) |
|
|
374 | (1) |
|
|
375 | (1) |
|
|
375 | (1) |
|
|
375 | (1) |
|
|
376 | (1) |
|
|
376 | (2) |
|
|
376 | (1) |
|
Log Server and Log Collector |
|
|
377 | (1) |
|
Log Server and Log Collector with Long-Term Storage |
|
|
378 | (1) |
|
|
378 | (1) |
|
|
378 | (1) |
|
|
379 | (2) |
|
|
381 | (20) |
|
|
381 | (1) |
|
|
381 | (5) |
|
|
382 | (1) |
|
|
383 | (1) |
|
Characteristics of a Cloud Infrastructure |
|
|
384 | (1) |
|
Standards? We Don't Need No Stinking Standards! |
|
|
385 | (1) |
|
|
386 | (4) |
|
|
388 | (2) |
|
Regulatory, Compliance, and Security Issues |
|
|
390 | (2) |
|
|
392 | (3) |
|
|
394 | (1) |
|
|
395 | (1) |
|
Pros and Cons of Cloud Logging |
|
|
396 | (1) |
|
Cloud Logging Provider Inventory |
|
|
396 | (1) |
|
|
396 | (2) |
|
|
398 | (3) |
|
Chapter 22 Log Standards and Future Trends |
|
|
401 | (12) |
|
|
401 | (1) |
|
Extrapolations of Today to the Future |
|
|
402 | (4) |
|
|
402 | (2) |
|
|
404 | (1) |
|
|
405 | (1) |
|
|
406 | (4) |
|
|
410 | (1) |
|
|
410 | (1) |
|
|
411 | (2) |
Index |
|
413 | |