Foreword |
|
xv | |
Preface |
|
xvii | |
Prologue: Imagine Data Mesh |
|
xxv | |
|
Part I What Is Data Mesh? |
|
|
|
1 Data Mesh in a Nutshell |
|
|
3 | (12) |
|
|
4 | (1) |
|
|
4 | (2) |
|
|
6 | (3) |
|
Principle of Domain Ownership |
|
|
6 | (1) |
|
Principle of Data as a Product |
|
|
7 | (1) |
|
Principle of the Self-Serve Data Platform |
|
|
8 | (1) |
|
Principle of Federated Computational Governance |
|
|
8 | (1) |
|
Interplay of the Principles |
|
|
9 | (1) |
|
Data Mesh Model at a Glance |
|
|
10 | (1) |
|
|
11 | (2) |
|
|
11 | (1) |
|
|
12 | (1) |
|
|
13 | (2) |
|
2 Principle of Domain Ownership |
|
|
15 | (14) |
|
A Brief Background on Domain-Driven Design |
|
|
17 | (1) |
|
Applying DDD's Strategic Design to Data |
|
|
18 | (2) |
|
|
20 | (4) |
|
Source-Aligned Domain Data |
|
|
21 | (2) |
|
|
23 | (1) |
|
Consumer-Aligned Domain Data |
|
|
24 | (1) |
|
Transition to Domain Ownership |
|
|
24 | (3) |
|
Push Data Ownership Upstream |
|
|
24 | (1) |
|
Define Multiple Connected Models |
|
|
25 | (1) |
|
Embrace the Most Relevant Domain Data: Don't Expect a Single Source of Truth |
|
|
26 | (1) |
|
Hide the Data Pipelines as Domains' Internal Implementation |
|
|
26 | (1) |
|
|
27 | (2) |
|
3 Principle of Data as a Product |
|
|
29 | (18) |
|
Applying Product Thinking to Data |
|
|
31 | (10) |
|
Baseline Usability Attributes of a Data Product |
|
|
33 | (8) |
|
Transition to Data as a Product |
|
|
41 | (4) |
|
Include Data Product Ownership in Domains |
|
|
42 | (1) |
|
Reframe the Nomenclature to Create Change |
|
|
42 | (1) |
|
Think of Data as a Product, Not a Mere Asset |
|
|
43 | (1) |
|
Establish a Trust-But-Verify Data Culture |
|
|
43 | (1) |
|
Join Data and Compute as One Logical Unit |
|
|
44 | (1) |
|
|
45 | (2) |
|
4 Principle of the Self-Serve Data Platform |
|
|
47 | (20) |
|
Data Mesh Platform: Compare and Contrast |
|
|
49 | (5) |
|
Serving Autonomous Domain-Oriented Teams |
|
|
51 | (1) |
|
Managing Autonomous and Interoperable Data Products |
|
|
51 | (1) |
|
A Continuous Platform of Operational and Analytical Capabilities |
|
|
52 | (1) |
|
Designed for a Generalist Majority |
|
|
52 | (1) |
|
Favoring Decentralized Technologies |
|
|
53 | (1) |
|
|
54 | (1) |
|
Data Mesh Platform Thinking |
|
|
54 | (8) |
|
Enable Autonomous Teams to Get Value from Data |
|
|
57 | (1) |
|
Exchange Value with Autonomous and Interoperable Data Products |
|
|
58 | (1) |
|
Accelerate Exchange of Value by Lowering the Cognitive Load |
|
|
59 | (1) |
|
|
60 | (2) |
|
Support a Culture of Embedded Innovation |
|
|
62 | (1) |
|
Transition to a Self-Serve Data Mesh Platform |
|
|
62 | (3) |
|
Design the APIs and Protocols First |
|
|
62 | (1) |
|
Prepare for Generalist Adoption |
|
|
63 | (1) |
|
Do an Inventory and Simplify |
|
|
63 | (1) |
|
Create Higher-Level APIs to Manage Data Products |
|
|
64 | (1) |
|
Build Experiences, Not Mechanisms |
|
|
64 | (1) |
|
Begin with the Simplest Foundation, Then Harvest to Evolve |
|
|
65 | (1) |
|
|
65 | (2) |
|
5 Principle of Federated Computational Governance |
|
|
67 | (28) |
|
Apply Systems Thinking to Data Mesh Governance |
|
|
69 | (6) |
|
Maintain Dynamic Equilibrium Between Domain Autonomy and Global Interoperability |
|
|
71 | (3) |
|
Embrace Dynamic Topology as a Default State |
|
|
74 | (1) |
|
Utilize Automation and the Distributed Architecture |
|
|
75 | (1) |
|
Apply Federation to the Governance Model |
|
|
75 | (8) |
|
|
77 | (1) |
|
|
78 | (3) |
|
|
81 | (1) |
|
|
82 | (1) |
|
Apply Computation to the Governance Model |
|
|
83 | (3) |
|
|
84 | (1) |
|
|
85 | (1) |
|
|
86 | (1) |
|
|
86 | (1) |
|
Transition to Federated Computational Governance |
|
|
86 | (3) |
|
Delegate Accountability to Domains |
|
|
86 | (1) |
|
Embed Policy Execution in Each Data Product |
|
|
87 | (1) |
|
Automate Enablement and Monitoring over Interventions |
|
|
87 | (1) |
|
|
88 | (1) |
|
Measure the Network Effect |
|
|
88 | (1) |
|
Embrace Change over Constancy |
|
|
88 | (1) |
|
|
89 | (6) |
|
|
|
|
95 | (10) |
|
Great Expectations of Data |
|
|
96 | (2) |
|
|
98 | (2) |
|
Scale: Encounter of a New Kind |
|
|
100 | (1) |
|
|
101 | (1) |
|
Approaching the Plateau of Return |
|
|
102 | (1) |
|
|
102 | (3) |
|
7 After the Inflection Point |
|
|
105 | (16) |
|
Respond Gracefully to Change in a Complex Business |
|
|
106 | (5) |
|
Align Business, Tech, and Now Analytical Data |
|
|
107 | (1) |
|
Close the Gap Between Analytical and Operational Data |
|
|
108 | (2) |
|
Localize Data Changes to Business Domains |
|
|
110 | (1) |
|
Reduce Accidental Complexity of Pipelines and Copying Data |
|
|
111 | (1) |
|
Sustain Agility in the Face of Growth |
|
|
111 | (4) |
|
Remove Centralized and Monolithic Bottlenecks |
|
|
112 | (1) |
|
Reduce Coordination of Data Pipelines |
|
|
112 | (1) |
|
Reduce Coordination of Data Governance |
|
|
113 | (2) |
|
|
115 | (1) |
|
Increase the Ratio of Value from Data to Investment |
|
|
115 | (2) |
|
Abstract Technical Complexity with a Data Platform |
|
|
116 | (1) |
|
Embed Product Thinking Everywhere |
|
|
116 | (1) |
|
|
116 | (1) |
|
|
117 | (4) |
|
8 Before the Inflection Point |
|
|
121 | (22) |
|
Evolution of Analytical Data Architectures |
|
|
121 | (5) |
|
First Generation: Data Warehouse Architecture |
|
|
122 | (1) |
|
Second Generation: Data Lake Architecture |
|
|
123 | (3) |
|
Third Generation: Multimodal Cloud Architecture |
|
|
126 | (1) |
|
Characteristics of Analytical Data Architecture |
|
|
126 | (11) |
|
|
128 | (4) |
|
Centralized Data Ownership |
|
|
132 | (1) |
|
|
133 | (4) |
|
|
137 | (6) |
|
Part III How to Design the Data Mesh Architecture |
|
|
|
9 The Logical Architecture |
|
|
143 | (28) |
|
Domain-Oriented Analytical Data Sharing Interfaces |
|
|
147 | (4) |
|
Operational Interface Design |
|
|
148 | (1) |
|
Analytical Data Interface Design |
|
|
149 | (1) |
|
Interdomain Analytical Data Dependencies |
|
|
149 | (2) |
|
Data Product as an Architecture Quantum |
|
|
151 | (9) |
|
A Data Product's Structural Components |
|
|
152 | (6) |
|
Data Product Data Sharing Interactions |
|
|
158 | (1) |
|
Data Discovery and Observability APIs |
|
|
159 | (1) |
|
The Multiplane Data Platform |
|
|
160 | (4) |
|
|
161 | (1) |
|
Data Infrastructure (Utility) Plane |
|
|
162 | (1) |
|
Data Product Experience Plane |
|
|
162 | (1) |
|
|
162 | (1) |
|
|
163 | (1) |
|
Embedded Computational Policies |
|
|
164 | (4) |
|
|
165 | (1) |
|
Data Product Computational Container |
|
|
166 | (1) |
|
|
167 | (1) |
|
|
168 | (3) |
|
10 The Multiplane Data Platform Architecture |
|
|
171 | (22) |
|
Design a Platform Driven by User Journeys |
|
|
174 | (1) |
|
Data Product Developer Journey |
|
|
175 | (10) |
|
Incept, Explore, Bootstrap, and Source |
|
|
177 | (3) |
|
Build, Test, Deploy, and Run |
|
|
180 | (3) |
|
Maintain, Evolve, and Retire |
|
|
183 | (2) |
|
Data Product Consumer Journey |
|
|
185 | (4) |
|
Incept, Explore, Bootstrap, Source |
|
|
188 | (1) |
|
|
188 | (1) |
|
Maintain, Evolve, and Retire |
|
|
189 | (1) |
|
|
189 | (4) |
|
Part IV How to Design the Data Product Architecture |
|
|
|
11 Design a Data Product by Affordances |
|
|
193 | (8) |
|
|
194 | (3) |
|
Data Product Architecture Characteristics |
|
|
197 | (1) |
|
Design Influenced by the Simplicity of Complex Adaptive Systems |
|
|
198 | (2) |
|
Emergent Behavior from Simple Local Rules |
|
|
198 | (1) |
|
|
199 | (1) |
|
|
200 | (1) |
|
12 Design Consuming, Transforming, and Serving Data |
|
|
201 | (32) |
|
|
201 | (16) |
|
|
201 | (3) |
|
Serve Data Design Properties |
|
|
204 | (12) |
|
|
216 | (1) |
|
|
217 | (9) |
|
Archetypes of Data Sources |
|
|
219 | (4) |
|
Locality of Data Consumption |
|
|
223 | (1) |
|
|
224 | (2) |
|
|
226 | (5) |
|
Programmatic Versus Nonprogrammatic Transformation |
|
|
226 | (2) |
|
Dataflow-Based Transformation |
|
|
228 | (1) |
|
|
229 | (1) |
|
Time-Variant Transformation |
|
|
229 | (1) |
|
|
230 | (1) |
|
|
231 | (2) |
|
13 Design Discovering, Understanding, and Composing Data |
|
|
233 | (22) |
|
Discover, Understand, Trust, and Explore |
|
|
233 | (11) |
|
Begin Discovery with Self-Registration |
|
|
236 | (1) |
|
|
236 | (1) |
|
Understand Semantic and Syntax Models |
|
|
237 | (1) |
|
Establish Trust with Data Guarantees |
|
|
238 | (3) |
|
Explore the Shape of Data |
|
|
241 | (1) |
|
|
242 | (1) |
|
Discover, Explore, and Understand Design |
|
|
242 | (2) |
|
|
244 | (8) |
|
Consume Data Design Properties |
|
|
245 | (1) |
|
Traditional Approaches to Data Composability |
|
|
246 | (4) |
|
|
250 | (2) |
|
|
252 | (3) |
|
14 Design Managing, Governing, and Observing Data |
|
|
255 | (16) |
|
|
255 | (3) |
|
|
256 | (1) |
|
Data Product Manifest Components |
|
|
257 | (1) |
|
|
258 | (4) |
|
|
259 | (1) |
|
|
260 | (2) |
|
Data and Policy Integration |
|
|
262 | (1) |
|
|
262 | (1) |
|
Observe, Debug, and Audit |
|
|
262 | (5) |
|
|
264 | (3) |
|
|
267 | (4) |
|
Part V How to Get Started |
|
|
|
15 Strategy and Execution |
|
|
271 | (32) |
|
Should You Adopt Data Mesh Today? |
|
|
271 | (4) |
|
Data Mesh as an Element of Data Strategy |
|
|
275 | (4) |
|
Data Mesh Execution Framework |
|
|
279 | (23) |
|
Business-Driven Execution |
|
|
280 | (5) |
|
End-to-End and Iterative Execution |
|
|
285 | (1) |
|
|
286 | (16) |
|
|
302 | (1) |
|
16 Organization and Culture |
|
|
303 | (30) |
|
|
305 | (2) |
|
|
307 | (3) |
|
|
308 | (2) |
|
|
310 | (2) |
|
|
311 | (1) |
|
|
311 | (1) |
|
|
312 | (12) |
|
Organization Structure Assumptions |
|
|
313 | (8) |
|
Discover Data Product Boundaries |
|
|
321 | (3) |
|
|
324 | (5) |
|
|
324 | (3) |
|
|
327 | (2) |
|
|
329 | (2) |
|
|
330 | (1) |
|
|
331 | (2) |
Index |
|
333 | |