Preface |
|
xiii | |
Part I. Data |
|
|
1 Organizing Data: Vantage, Domain, Action, and Validity |
|
|
3 | (16) |
|
|
5 | (1) |
|
|
6 | (2) |
|
|
8 | (1) |
|
Actions: What a Sensor Does with Data |
|
|
9 | (2) |
|
|
11 | (2) |
|
|
13 | (1) |
|
|
14 | (1) |
|
|
15 | (1) |
|
|
15 | (1) |
|
Attacker and Attack Issues |
|
|
16 | (1) |
|
|
16 | (3) |
|
2 Vantage: Understanding Sensor Placement in Networks |
|
|
19 | (16) |
|
The Basics of Network Layering |
|
|
19 | (3) |
|
Network Layers and Vantage |
|
|
22 | (4) |
|
Network Layers and Addressing |
|
|
26 | (1) |
|
|
27 | (1) |
|
IPv4 Format and Addresses |
|
|
28 | (1) |
|
IPv6 Format and Addresses |
|
|
28 | (1) |
|
Validity Challenges from Middlebox Network Data |
|
|
29 | (5) |
|
|
34 | (1) |
|
3 Sensors in the Network Domain |
|
|
35 | (28) |
|
|
36 | (1) |
|
|
36 | (1) |
|
Limiting the Data Captured from Each Packet |
|
|
37 | (1) |
|
Filtering Specific Types of Packets |
|
|
37 | (4) |
|
What If It's Not Ethernet? |
|
|
41 | (1) |
|
|
41 | (1) |
|
NetFlow v5 Formats and Fields |
|
|
42 | (2) |
|
NetFlow Generation and Collection |
|
|
44 | (1) |
|
|
44 | (1) |
|
|
45 | (1) |
|
|
46 | (4) |
|
Improving IDS Performance |
|
|
50 | (1) |
|
|
51 | (1) |
|
|
52 | (5) |
|
|
57 | (1) |
|
|
58 | (1) |
|
Middlebox Logs and Their Impact |
|
|
59 | (1) |
|
|
60 | (1) |
|
|
60 | (1) |
|
|
61 | (1) |
|
|
61 | (2) |
|
4 . Data in the Service Domain |
|
|
63 | (14) |
|
|
63 | (2) |
|
Logfiles as the Basis for Service Data |
|
|
65 | (1) |
|
Accessing and Manipulating Logfiles |
|
|
65 | (2) |
|
|
67 | (1) |
|
The Characteristics of a Good Log Message |
|
|
67 | (3) |
|
Existing Logfiles and How to Manipulate Them |
|
|
70 | (2) |
|
|
72 | (3) |
|
|
75 | (2) |
|
5 Sensors in the Service Domain |
|
|
77 | (14) |
|
Representative Logfile Formats |
|
|
78 | (1) |
|
|
78 | (4) |
|
Simple Mail Transfer Protocol (SMTP) |
|
|
82 | (1) |
|
|
82 | (2) |
|
Microsoft Exchange: Message Tracking Logs |
|
|
84 | (1) |
|
Additional Useful Logfiles |
|
|
85 | (1) |
|
|
85 | (1) |
|
LDAP and Directory Services |
|
|
86 | (1) |
|
File Transfer, Storage, and Databases |
|
|
86 | (1) |
|
Logfile Transport: Transfers, Syslog, and Message Queues |
|
|
87 | (1) |
|
Transfer and Logfile Rotation |
|
|
87 | (1) |
|
|
87 | (2) |
|
|
89 | (2) |
|
6 Data and Sensors in the Host Domain |
|
|
91 | (16) |
|
A Host: From the Network's View |
|
|
92 | (1) |
|
|
93 | (3) |
|
The Host: Tracking Identity |
|
|
96 | (2) |
|
|
98 | (1) |
|
|
98 | (3) |
|
|
101 | (2) |
|
Historical Data: Commands and Logins |
|
|
103 | (1) |
|
Other Data and Sensors: HIPS and AV |
|
|
104 | (1) |
|
|
105 | (2) |
|
7 Data and Sensors in the Active Domain |
|
|
107 | (16) |
|
Discovery, Assessment, and Maintenance |
|
|
107 | (1) |
|
Discovery: ping, traceroute, netcat, and Half of nmap |
|
|
108 | (1) |
|
Checking Connectivity: Using ping to Connect to an Address |
|
|
108 | (2) |
|
|
110 | (2) |
|
Using nc as a Swiss Army Multitool |
|
|
112 | (1) |
|
nmap Scanning for Discovery |
|
|
113 | (2) |
|
Assessment: nmap, a Bunch of Clients, and a Lot of Repositories |
|
|
115 | (1) |
|
Basic Assessment with nmap |
|
|
115 | (4) |
|
Using Active Vantage Data for Verification |
|
|
119 | (1) |
|
|
120 | (3) |
Part II. Tools |
|
|
8 Getting Data in One Place |
|
|
123 | (14) |
|
|
125 | (1) |
|
|
126 | (1) |
|
|
127 | (2) |
|
|
129 | (1) |
|
|
130 | (1) |
|
|
130 | (1) |
|
Log Data and the CRUD Paradigm |
|
|
131 | (2) |
|
A Brief Introduction to NoSQL Systems |
|
|
133 | (3) |
|
|
136 | (1) |
|
|
137 | (36) |
|
What Is SiLK and How Does It Work? |
|
|
137 | (1) |
|
Acquiring and Installing SiLK |
|
|
138 | (1) |
|
|
139 | (1) |
|
Choosing and Formatting Output Field Manipulation: rwcut |
|
|
139 | (5) |
|
Basic Field Manipulation: rwfilter |
|
|
144 | (1) |
|
|
145 | (1) |
|
|
146 | (1) |
|
|
147 | (1) |
|
|
148 | (1) |
|
|
148 | (2) |
|
|
150 | (1) |
|
Miscellaneous Filtering Options and Some Hacks |
|
|
151 | (1) |
|
rwfileinfo and Provenance |
|
|
152 | (2) |
|
Combining Information Flows: rwcount |
|
|
154 | (3) |
|
|
157 | (4) |
|
|
161 | (1) |
|
|
162 | (1) |
|
|
163 | (1) |
|
|
163 | (2) |
|
|
165 | (1) |
|
|
166 | (2) |
|
|
168 | (1) |
|
|
169 | (1) |
|
|
170 | (1) |
|
|
171 | (2) |
|
10 Reference and Lookup: Tools for Figuring Out Who Someone Is |
|
|
173 | (26) |
|
MAC and Hardware Addresses |
|
|
174 | (2) |
|
|
176 | (1) |
|
IPv4 Addresses, Their Structure, and Significant Addresses |
|
|
176 | (2) |
|
IPv6 Addresses, Their Structure, and Significant Addresses |
|
|
178 | (2) |
|
IP Intelligence: Geolocation and Demographics |
|
|
180 | (1) |
|
|
181 | (1) |
|
|
181 | (2) |
|
Forward DNS Querying Using dig |
|
|
183 | (8) |
|
|
191 | (1) |
|
Using whois to Find Ownership |
|
|
192 | (3) |
|
|
195 | (2) |
|
|
197 | (1) |
|
|
197 | (1) |
|
Scanning Repositories, Shodan et al |
|
|
198 | (1) |
|
|
198 | (1) |
Part III. Analytics An Overview of Attacker Behavior |
|
199 | (188) |
|
|
202 | (1) |
|
11 Exploratory Data Analysis and Visualization |
|
|
203 | (32) |
|
The Goal of EDA: Applying Analysis |
|
|
205 | (2) |
|
|
207 | (1) |
|
Variables and Visualization |
|
|
208 | (1) |
|
|
209 | (1) |
|
|
210 | (2) |
|
Bar Plots (Not Pie Charts) |
|
|
212 | (1) |
|
The Five-Number Summary and the Boxplot |
|
|
212 | (2) |
|
|
214 | (1) |
|
|
215 | (1) |
|
|
215 | (2) |
|
Multivariate Visualization |
|
|
217 | (1) |
|
Other Visualizations and Their Role |
|
|
218 | (4) |
|
Operationalizing Security Visualization |
|
|
222 | (6) |
|
|
228 | (1) |
|
|
228 | (1) |
|
Simply Visualizing: Projected Values and QQ Plots |
|
|
228 | (3) |
|
|
231 | (2) |
|
|
233 | (2) |
|
|
235 | (18) |
|
|
235 | (3) |
|
|
238 | (1) |
|
|
239 | (3) |
|
|
242 | (1) |
|
|
242 | (1) |
|
|
243 | (1) |
|
Splitting Along Delimiters |
|
|
243 | (1) |
|
|
244 | (3) |
|
Techniques for Text Analysis |
|
|
247 | (1) |
|
|
247 | (1) |
|
|
247 | (1) |
|
|
248 | (1) |
|
|
248 | (2) |
|
Entropy and Compressibility |
|
|
250 | (1) |
|
|
251 | (1) |
|
|
252 | (1) |
|
|
253 | (20) |
|
Fumbling: Misconfiguration, Automation, and Scanning |
|
|
253 | (1) |
|
|
254 | (1) |
|
|
254 | (1) |
|
|
255 | (1) |
|
|
255 | (2) |
|
IP Fumbling: Dark Addresses and Spread |
|
|
257 | (2) |
|
TCP Fumbling: Failed Sessions |
|
|
259 | (5) |
|
ICMP Messages and Fumbling |
|
|
264 | (1) |
|
Fumbling at the Service Level |
|
|
265 | (1) |
|
|
265 | (2) |
|
|
267 | (1) |
|
|
267 | (1) |
|
Detecting and Analyzing Fumbling |
|
|
268 | (1) |
|
|
268 | (2) |
|
Forensic Analysis of Fumbling |
|
|
270 | (1) |
|
Engineering a Network to Take Advantage of Fumbling |
|
|
271 | (2) |
|
|
273 | (26) |
|
The Workday and Its Impact on Network Traffic Volume |
|
|
273 | (3) |
|
|
276 | (3) |
|
|
279 | (3) |
|
|
282 | (3) |
|
DDoS, Flash Crowds, and Resource Exhaustion |
|
|
285 | (1) |
|
DDoS and Routing Infrastructure |
|
|
286 | (6) |
|
Applying Volume and Locality Analysis |
|
|
292 | (1) |
|
|
292 | (3) |
|
|
295 | (1) |
|
Using Beaconing as an Alarm |
|
|
295 | (1) |
|
Using Locality as an Alarm |
|
|
295 | (1) |
|
|
296 | (1) |
|
|
296 | (3) |
|
|
299 | (18) |
|
Graph Attributes: What Is a Graph? |
|
|
299 | (4) |
|
Labeling, Weight, and Paths |
|
|
303 | (5) |
|
Components and Connectivity |
|
|
308 | (1) |
|
|
309 | (2) |
|
|
311 | (1) |
|
Using Component Analysis as an Alarm |
|
|
311 | (1) |
|
Using Centrality Analysis for Forensics |
|
|
312 | (1) |
|
Using Breadth-First Searches Forensically |
|
|
313 | (2) |
|
Using Centrality Analysis for Engineering |
|
|
315 | (1) |
|
|
315 | (2) |
|
|
317 | (12) |
|
Insider Threat Versus Other Classes of Attacks |
|
|
318 | (3) |
|
|
321 | (1) |
|
|
322 | (1) |
|
Data Theft and Exfiltration |
|
|
322 | (1) |
|
|
323 | (1) |
|
|
323 | (1) |
|
Insider Threat Data: Logistics and Collection |
|
|
324 | (1) |
|
Applying Sector-Based Workflow to Insider Threat |
|
|
324 | (2) |
|
|
326 | (1) |
|
Keeping Track of User Identity |
|
|
326 | (1) |
|
|
327 | (2) |
|
17 On Threat Intelligence |
|
|
329 | (10) |
|
Defining Threat Intelligence |
|
|
329 | (1) |
|
|
330 | (3) |
|
Creating a Threat Intelligence Program |
|
|
333 | (1) |
|
|
333 | (2) |
|
Starting with Free Sources |
|
|
335 | (1) |
|
|
335 | (1) |
|
|
335 | (2) |
|
Brief Remarks on Creating Threat Intelligence |
|
|
337 | (1) |
|
|
337 | (2) |
|
18 Application Identification |
|
|
339 | (16) |
|
Mechanisms for Application Identification |
|
|
339 | (1) |
|
|
340 | (4) |
|
Application Identification by Banner Grabbing |
|
|
344 | (3) |
|
Application Identification by Behavior |
|
|
347 | (4) |
|
Application Identification by Subsidiary Site |
|
|
351 | (1) |
|
Application Banners: Identifying and Classifying |
|
|
351 | (1) |
|
|
351 | (1) |
|
Web Client Banners: The User-Agent String |
|
|
352 | (2) |
|
|
354 | (1) |
|
|
355 | (18) |
|
Creating an Initial Network Inventory and Map |
|
|
355 | (1) |
|
Creating an Inventory: Data, Coverage, and Files |
|
|
356 | (2) |
|
Phase I: The First Three Questions |
|
|
358 | (2) |
|
Phase II: Examining the IP Space |
|
|
360 | (5) |
|
Phase III: Identifying Blind and Confusing Traffic |
|
|
365 | (3) |
|
Phase IV: Identifying Clients and Servers |
|
|
368 | (3) |
|
Identifying Sensing and Blocking Infrastructure |
|
|
371 | (1) |
|
Updating the Inventory: Toward Continuous Audit |
|
|
371 | (1) |
|
|
372 | (1) |
|
|
373 | (12) |
|
Ops Environments: An Overview |
|
|
373 | (1) |
|
|
374 | (1) |
|
|
375 | (2) |
|
|
377 | (2) |
|
|
379 | (1) |
|
|
380 | (2) |
|
|
382 | (1) |
|
|
383 | (1) |
|
|
384 | (1) |
|
|
385 | (2) |
Index |
|
387 | |