Muutke küpsiste eelistusi

Concurrent Data Processing in Elixir: Fast, Resilient Applications with OTP, GenStage, Flow, and Broadway [Pehme köide]

  • Formaat: Paperback / softback, 170 pages
  • Ilmumisaeg: 14-Oct-2021
  • Kirjastus: The Pragmatic Programmers
  • ISBN-10: 1680508199
  • ISBN-13: 9781680508192
  • Formaat: Paperback / softback, 170 pages
  • Ilmumisaeg: 14-Oct-2021
  • Kirjastus: The Pragmatic Programmers
  • ISBN-10: 1680508199
  • ISBN-13: 9781680508192

Learn different ways of writing concurrent code in Elixir and increase your application's performance, without sacrificing scalability or fault-tolerance. Most projects benefit from running background tasks and processing data concurrently, but the world of OTP and various libraries can be challenging. Which Supervisor and what strategy to use? What about GenServer? Maybe you need back-pressure, but is GenStage, Flow, or Broadway a better choice? You will learn everything you need to know to answer these questions, start building highly concurrent applications in no time, and write code that's not only fast, but also resilient to errors and easy to scale.

Whether you are building a high-frequency stock trading application or a consumer web app, you need to know how to leverage concurrency to build applications that are fast and efficient. Elixir and the OTP offer a range of powerful tools, and this guide will show you how to choose the best tool for each job, and use it effectively to quickly start building highly concurrent applications.

Learn about Tasks, supervision trees, and the different types of Supervisors available to you. Understand why processes and process linking are the building blocks of concurrency in Elixir. Get comfortable with the OTP and use the GenServer behaviour to maintain process state for long-running jobs. Easily scale the number of running processes using the Registry. Handle large volumes of data and traffic spikes with GenStage, using back-pressure to your advantage. Create your first multi-stage data processing pipeline using producer, consumer, and producer-consumer stages. Process large collections with Flow, using MapReduce and more in parallel. Thanks to Broadway, you will see how easy it is to integrate with popular message broker systems, or even existing GenStage producers.

Start building the high-performance and fault-tolerant applications Elixir is famous for today.

What You Need:

You'll need Elixir 1.9+ and Erlang/OTP 22+ installed on a Mac OS X, Linux, or Windows machine.

Foreword vii
Acknowledgments ix
Introduction xi
1 Easy Concurrency with the Task Module
1(24)
Introducing the Task Module
2(1)
Creating Our Playground
3(1)
Starting Tasks and Retrieving Results
4(7)
Managing Series of Tasks
11(4)
Linking Processes
15(2)
Meeting the Supervisor
17(5)
Understanding Let It Crash
22(1)
Wrapping Up
23(2)
2 Long-Running Processes Using GenServer
25(32)
Starting with a Basic GenServer
26(1)
GenServer Callbacks In Depth
27(9)
Building a Job-Processing System
36(5)
Introducing DynamicSupervisor
41(4)
Implementing a Supervisor
45(4)
Naming Processes Using the Registry
49(5)
Inspecting Supervisors at Runtime
54(2)
Wrapping Up
56(1)
3 Data-Processing Pipelines with GenStage
57(32)
Understanding Back-Pressure
58(1)
Introducing GenStage
58(4)
Building Your Data-Processing Pipeline
62(12)
Adding Concurrency with ConsumerSupervisor
74(4)
Creating Multi-Stage Data Pipelines
78(6)
Choosing the Right Dispatcher
84(3)
Wrapping Up
87(2)
4 Processing Collections with Flow
89(24)
Analyzing Airports Data
89(2)
Creating Flows and Reading Files
91(6)
Performing Reduce Concurrently with Partitions
97(6)
Using Windows and Triggers
103(5)
Adding Flow to a GenStage Pipeline
108(3)
Wrapping Up
111(2)
5 Data-Ingestion Pipelines with Broadway
113(36)
Processing Ticketing Events
114(2)
Broadway Callbacks In Depth
116(11)
Batching Messages
127(11)
Using GenStage Producers
138(6)
Wrapping Up
144(3)
Bibliography
147(2)
Index 149
Svilen Gospodinov is a software engineer with a technical leadership and entrepreneurship background. As co-founder and CTO at Heresy, a workflow and analytics platform for Sales, he adopted Elixir from the very beginning and never looked back. He likes building products, writing, working on open source, and helping others.