Preface |
|
ix | |
|
1 Overview of Machine Learning Systems |
|
|
1 | (24) |
|
When to Use Machine Learning |
|
|
3 | (9) |
|
Machine Learning Use Cases |
|
|
9 | (3) |
|
Understanding Machine Learning Systems |
|
|
12 | (11) |
|
Machine Learning in Research Versus in Production |
|
|
12 | (10) |
|
Machine Learning Systems Versus Traditional Software |
|
|
22 | (1) |
|
|
23 | (2) |
|
2 Introduction to Machine Learning Systems Design |
|
|
25 | (24) |
|
Business and ML Objectives |
|
|
26 | (3) |
|
Requirements for ML Systems |
|
|
29 | (3) |
|
|
29 | (1) |
|
|
30 | (1) |
|
|
31 | (1) |
|
|
31 | (1) |
|
|
32 | (3) |
|
|
35 | (8) |
|
|
36 | (4) |
|
|
40 | (3) |
|
|
43 | (3) |
|
|
46 | (3) |
|
3 Data Engineering Fundamentals |
|
|
49 | (32) |
|
|
50 | (3) |
|
|
53 | (5) |
|
|
54 | (1) |
|
Row-Major Versus Column-Major Format |
|
|
54 | (3) |
|
Text Versus Binary Format |
|
|
57 | (1) |
|
|
58 | (9) |
|
|
59 | (4) |
|
|
63 | (3) |
|
Structured Versus Unstructured Data |
|
|
66 | (1) |
|
Data Storage Engines and Processing |
|
|
67 | (5) |
|
Transactional and Analytical Processing |
|
|
67 | (3) |
|
ETL: Extract, Transform, and Load |
|
|
70 | (2) |
|
|
72 | (6) |
|
Data Passing Through Databases |
|
|
72 | (1) |
|
Data Passing Through Services |
|
|
73 | (1) |
|
Data Passing Through Real-Time Transport |
|
|
74 | (4) |
|
Batch Processing Versus Stream Processing |
|
|
78 | (1) |
|
|
79 | (2) |
|
|
81 | (38) |
|
|
82 | (6) |
|
|
83 | (1) |
|
|
84 | (1) |
|
|
84 | (1) |
|
|
85 | (1) |
|
|
86 | (1) |
|
|
87 | (1) |
|
|
88 | (14) |
|
|
88 | (3) |
|
|
91 | (3) |
|
Handling the Lack of Labels |
|
|
94 | (8) |
|
|
102 | (11) |
|
Challenges of Class Imbalance |
|
|
103 | (2) |
|
|
105 | (8) |
|
|
113 | (5) |
|
Simple Label-Preserving Transformations |
|
|
114 | (1) |
|
|
114 | (2) |
|
|
116 | (2) |
|
|
118 | (1) |
|
|
119 | (30) |
|
Learned Features Versus Engineered Features |
|
|
120 | (3) |
|
Common Feature Engineering Operations |
|
|
123 | (12) |
|
|
123 | (3) |
|
|
126 | (2) |
|
|
128 | (1) |
|
Encoding Categorical Features |
|
|
129 | (3) |
|
|
132 | (1) |
|
Discrete and Continuous Positional Embeddings |
|
|
133 | (2) |
|
|
135 | (6) |
|
Common Causes for Data Leakage |
|
|
137 | (3) |
|
|
140 | (1) |
|
Engineering Good Features |
|
|
141 | (5) |
|
|
142 | (2) |
|
|
144 | (2) |
|
|
146 | (3) |
|
6 Model Development and Offline Evaluation |
|
|
149 | (42) |
|
Model Development and Training |
|
|
150 | (28) |
|
|
150 | (6) |
|
|
156 | (6) |
|
Experiment Tracking and Versioning |
|
|
162 | (6) |
|
|
168 | (4) |
|
|
172 | (6) |
|
|
178 | (10) |
|
|
179 | (2) |
|
|
181 | (7) |
|
|
188 | (3) |
|
7 Model Deployment and Prediction Service |
|
|
191 | (34) |
|
Machine Learning Deployment Myths |
|
|
194 | (3) |
|
Myth 1 You Only Deploy One or Two ML Models at a Time |
|
|
194 | (1) |
|
Myth 2 If We Don't Do Anything, Model Performance Remains the Same |
|
|
195 | (1) |
|
Myth 3 You Won't Need to Update Your Models as Much |
|
|
196 | (1) |
|
Myth 4 Most ML Engineers Don't Need to Worry About Scale |
|
|
196 | (1) |
|
Batch Prediction Versus Online Prediction |
|
|
197 | (9) |
|
From Batch Prediction to Online Prediction |
|
|
201 | (2) |
|
Unifying Batch Pipeline and Streaming Pipeline |
|
|
203 | (3) |
|
|
206 | (6) |
|
|
206 | (2) |
|
|
208 | (1) |
|
|
208 | (1) |
|
|
209 | (3) |
|
ML on the Cloud and on the Edge |
|
|
212 | (11) |
|
Compiling and Optimizing Models for Edge Devices |
|
|
214 | (8) |
|
|
222 | (1) |
|
|
223 | (2) |
|
8 Data Distribution Shifts and Monitoring |
|
|
225 | (38) |
|
Causes of ML System Failures |
|
|
226 | (11) |
|
|
227 | (2) |
|
|
229 | (8) |
|
|
237 | (13) |
|
Types of Data Distribution Shifts |
|
|
237 | (4) |
|
General Data Distribution Shifts |
|
|
241 | (1) |
|
Detecting Data Distribution Shifts |
|
|
242 | (6) |
|
Addressing Data Distribution Shifts |
|
|
248 | (2) |
|
Monitoring and Observability |
|
|
250 | (11) |
|
|
251 | (5) |
|
|
256 | (3) |
|
|
259 | (2) |
|
|
261 | (2) |
|
9 Continual Learning and Test in Production |
|
|
263 | (30) |
|
|
264 | (17) |
|
Stateless Retraining Versus Stateful Training |
|
|
265 | (3) |
|
|
268 | (2) |
|
Continual Learning Challenges |
|
|
270 | (4) |
|
Four Stages of Continual Learning |
|
|
274 | (5) |
|
How Often to Update Your Models |
|
|
279 | (2) |
|
|
281 | (10) |
|
|
282 | (1) |
|
|
283 | (2) |
|
|
285 | (1) |
|
|
285 | (2) |
|
|
287 | (4) |
|
|
291 | (2) |
|
10 Infrastructure and Tooling for MLOps |
|
|
293 | (38) |
|
|
297 | (5) |
|
Public Cloud Versus Private Data Centers |
|
|
300 | (2) |
|
|
302 | (9) |
|
|
303 | (3) |
|
Standardizing Dev Environments |
|
|
306 | (2) |
|
From Dev to Prod: Containers |
|
|
308 | (3) |
|
|
311 | (8) |
|
Cron, Schedulers, and Orchestrators |
|
|
311 | (3) |
|
Data Science Workflow Management |
|
|
314 | (5) |
|
|
319 | (8) |
|
|
320 | (1) |
|
|
321 | (4) |
|
|
325 | (2) |
|
|
327 | (2) |
|
|
329 | (2) |
|
11 The Human Side of Machine Learning |
|
|
331 | (24) |
|
|
331 | (3) |
|
Ensuring User Experience Consistency |
|
|
332 | (1) |
|
Combatting "Mosdy Correct" Predictions |
|
|
332 | (2) |
|
|
334 | (1) |
|
|
334 | (5) |
|
Cross-functional Teams Collaboration |
|
|
335 | (1) |
|
End-to-End Data Scientists |
|
|
335 | (4) |
|
|
339 | (14) |
|
Irresponsible AI: Case Studies |
|
|
341 | (6) |
|
A Framework for Responsible AI |
|
|
347 | (6) |
|
|
353 | (2) |
Epilogue |
|
355 | (2) |
Index |
|
357 | |