Muutke küpsiste eelistusi

E-raamat: Logging in Action: With Fluentd, Kubernetes and more

4.10/5 (20 hinnangut Goodreads-ist)
  • Formaat: 392 pages
  • Ilmumisaeg: 10-May-2022
  • Kirjastus: Manning Publications
  • Keel: eng
  • ISBN-13: 9781638355670
Teised raamatud teemal:
  • Formaat - EPUB+DRM
  • Hind: 51,64 €*
  • * hind on lõplik, st. muud allahindlused enam ei rakendu
  • Lisa ostukorvi
  • Lisa soovinimekirja
  • See e-raamat on mõeldud ainult isiklikuks kasutamiseks. E-raamatuid ei saa tagastada.
Logging in Action: With Fluentd, Kubernetes and moreSuurem pilt
  • Formaat: 392 pages
  • Ilmumisaeg: 10-May-2022
  • Kirjastus: Manning Publications
  • Keel: eng
  • ISBN-13: 9781638355670
Teised raamatud teemal:

DRM piirangud

  • Kopeerimine (copy/paste):

    ei ole lubatud

  • Printimine:

    ei ole lubatud

  • Kasutamine:

    Digitaalõiguste kaitse (DRM)
    Kirjastus on väljastanud selle e-raamatu krüpteeritud kujul, mis tähendab, et selle lugemiseks peate installeerima spetsiaalse tarkvara. Samuti peate looma endale  Adobe ID Rohkem infot siin. E-raamatut saab lugeda 1 kasutaja ning alla laadida kuni 6'de seadmesse (kõik autoriseeritud sama Adobe ID-ga).

    Vajalik tarkvara
    Mobiilsetes seadmetes (telefon või tahvelarvuti) lugemiseks peate installeerima selle tasuta rakenduse: PocketBook Reader (iOS / Android)

    PC või Mac seadmes lugemiseks peate installima Adobe Digital Editionsi (Seeon tasuta rakendus spetsiaalselt e-raamatute lugemiseks. Seda ei tohi segamini ajada Adober Reader'iga, mis tõenäoliselt on juba teie arvutisse installeeritud )

    Seda e-raamatut ei saa lugeda Amazon Kindle's. 

"A great book to introduce developers to the essential role that logging plays in software design and support." - Trent Whiteley

Logging in Action teaches you how to make your log processing a real asset for your application, all with free and open source tools. You'll use the powerful log management tool Fluentd to solve common log problems, and learn how proper log management can improve performance and make management of software solutions easier. Through useful examples like sending log driven events to Slack, you'll get hands-on experience applying structure to your unstructured data.

about the technology Fluentd is a powerful log management tool that seamlessly handles messy logging data, from operational errors, to application events, and security events. It decouples log data, such as SNMP or slow database queries, from backend systems and easily sends it where it needs to gothanks to 500+ flexible plugins covering all major services.

about the book Logging in Action is a guide to streamlining your log processing, turning logged data into a tool that will boost your application's performance. It's packed with tips and tricks on how proper log management can improve your application's security, speed, and cost-effectiveness. You'll master the basics of the Fluentd management tool, then take a peek behind the scenes at how different configurations will impact and improve the way your systems function. By the time you're done, you'll be able to reliably configure Fluentd so that it can easily scale to handle even the largest data loads.

what's inside Deploy Fluentd and Fluent Bit into varied environments Configure Fluentd and Fluent Bit to solve common log problems Fluentd for microservices Connect a custom log source or destination with Fluentd's extensibility framework Create a custom plugin for niche problems Logging best practices and common pitfalls

about the reader For developers and operations people familiar with application logging or infrastructure monitor logs.

about the author Phil Wilkins has spent over 25 years in the software industry. He works as a Technology Evangelist for Capgemini, specializing in cloud integration, APIs, and related technologies. Phil is TOGAF certified, and recognized by Oracle as an Ace Director (independent technology advocate) for his contributions to the integration and PaaS community.

Arvustused

"A great book to introduce developers to the essential role that logging plays in software design and support."  Trent Whiteley   "A comprehensive introduction to the Fluentd realm with valuable hands-on exercises." Michal Rutka   "A great guide to getting started with Fluentd and using it effectively in your operations." George Thomas   "Unified Logging with Fluentd is a text that takes you by the hand and explains step by step how to use Fluentd, covering all the key issues in a structured and very pragmatic way." Andrea C. Granata   "A great in-depth look at Fluentd, its applications, and its plugins." Joel Holmes

Foreword xiii
Preface xvi
Acknowledgments xviii
About this Book xx
About the Author xxv
About the cover illustration xxvi
Part 1 From zero to "Hello World"
1 Introduction to Fluentd
3(27)
1.1 Elevator pitch for Fluentd
4(2)
What is a log event?
4(1)
Fluentd compared to middleware
5(1)
1.2 Why do we produce logs?
6(3)
1.3 Evolving ideas
9(3)
Four golden signals
9(1)
Three pillars of observability
10(2)
1.4 Log unification
12(2)
Unifying logs vs. log analytics
14(1)
1.5 Software stacks
14(4)
ELK stack
15(1)
Comparing Fluentd and Logstash
16(1)
The relationship between Fluentd and Fluent Bit
17(1)
The relationship between Logstash and Beats
18(1)
1.6 Log routing as a vehicle for security
18(1)
1.7 Log event life cycle
19(1)
1.8 Evolution of Fluentd
20(3)
Treasure Data
20(2)
CNCF
22(1)
Relationship to major cloud vendors PaaS/IaaS
22(1)
1.9 Where can Fluentd and Fluent Bit be used?
23(1)
Platform constraints
23(1)
1.10 Fluentd UI-based editing
24(2)
1.11 Plugins
26(1)
1.12 How Fluentd can be used to make operational tasks easier
27(3)
Actionable log events
27(1)
Making logs more meaningful
27(1)
Polyglot environments
28(1)
Multiple targets
28(1)
Controlling log data costs
28(1)
Logs to metrics
28(1)
Rapid operational consolidation
29(1)
2 Concepts, architecture, and deployment of Fluentd
30(35)
2.1 Architecture and core concepts
30(8)
The makeup of a log event
31(1)
Handling time
32(1)
Architecture of Fluentd
33(2)
Fluent configuration execution order
35(1)
Directives
36(2)
Putting timing requirements into action
38(1)
2.2 Deployment of Fluentd
38(8)
Deploying Fluentd for the book's examples
39(1)
Deployment considerations for Fluentd
39(1)
Fluentd minimum footprint
40(1)
Simple deployment of Ruby
40(1)
Simple deployment of Fluentd
41(3)
Deploying a log generator
44(2)
Installing Postman
46(1)
2.3 Bringing Fluentd to life with "Hello World"
46(4)
"Hello World" scenario
46(1)
"Hello World" configuration
47(1)
Starting Fluentd
47(3)
2.4 "Hello World" with Fluent Bit
50(5)
Starting Fluent Bit
50(2)
Alternate Fluent Bit startup options
52(1)
Fluent Bit configuration file comparison
53(1)
Fluent Bit configuration file in detail
54(1)
Putting the dummy plugin into action
55(1)
2.5 Fluentd deployment with Kubernetes and containers
55(4)
Fluentd DaemonSet
56(2)
Dockerized Fluentd
58(1)
2.6 Using Fluentd UI
59(6)
Installing Fluentd with UI
59(6)
Part 2 Fluentd in depth
65(100)
3 Using Fluentd to capture log events
67(26)
3.1 Dry running to check a configuration
68(1)
Putting validating Fluentd configuration into action
69(1)
3.2 Reading log files
69(9)
Putting the adaption of a Fluentd configuration to Fluent Bit into action
70(1)
Rereading and resuming reading of log files
71(1)
Configuration considerations for tracking position
71(1)
Wildcards in the path attribute
72(1)
Expressing time
73(1)
Controlling the impact of wildcards in filenames
73(2)
Replacing wildcards with delimited lists in action
75(1)
Handling log rotation
75(3)
3.3 Self-monitoring
78(3)
HTTP interface check
78(3)
3.4 Imposing structure on log events
81(12)
Standard parsers
81(5)
Third-party parsers
86(1)
Applying a Regex parser to a complex log
86(5)
Putting parser configuration into action
91(2)
4 Using Fluentd to output log events
93(26)
4.1 File output plugin
94(9)
Basic file output
94(1)
Basics of buffering
95(3)
Chunks and Controlling Buffering
98(4)
Retry and backoff
102(1)
Putting configuring buffering size settings into action
103(1)
4.2 Output formatting options
103(3)
out_file
104(1)
json
104(1)
Itsv
104(1)
csv
104(1)
msgpack
105(1)
Applying formatters
105(1)
Putting JSON formatter configuration into action
106(1)
4.3 Sending log events to MongoDB
106(5)
Deploying MongoDB Fluentd plugin
107(1)
Configuring the Mongo output plugin for Fluentd
108(3)
Putting MongoDB connection configuration strings into action
111(1)
4.4 Actionable log events
111(2)
Actionable log events through service invocation
112(1)
Actionable through user interaction tools
112(1)
4.5 Slack to demonstrate the social output
113(4)
Handling tokens and credentials more carefully
115(1)
Externalizing Slack configuration attributes in action
116(1)
4.6 The right tool for the right job
117(2)
5 Routing log events
119(26)
5.1 Reaching multiple outputs by copying
120(6)
Copy by reference or by value
122(2)
Handling errors when copying
124(2)
5.2 Configuration reuse and extension through inclusion
126(4)
Place holding with null output
129(1)
Putting inclusions with a MongoDB output into action
129(1)
5.3 Injecting context into log events
130(2)
Extraction of values
131(1)
5.4 Tag-based routing
132(4)
Using exec output plugin
135(1)
Putting tag naming conventions into action
135(1)
Putting dynamic tagging with extract into action
136(1)
5.5 Tag plugins
136(1)
5.6 Labels: Taking tags to a new level
137(8)
Using a stdout filter to see what is happening
137(1)
Illustrating label and tag routing
137(2)
Connecting pipelines
139(2)
Label sequencing
141(2)
Special labels
143(1)
Putting a common pipeline into action
144(1)
6 Filtering and extrapolation
145(20)
6.1 Application of filters
146(2)
All is well events do not need to be distributed
146(1)
Spotting the needle in a haystack
147(1)
False urgency
147(1)
Releveling
147(1)
Unimplemented housekeeping
148(1)
6.2 Why change log events?
148(2)
Easier to process meaning downstream
148(1)
Add context
148(1)
Record when we have reacted to a log event
148(1)
Data redaction/masking
149(1)
6.3 Applying filters and parsers
150(7)
Filter plugins
150(1)
Applying grep filters
150(3)
Changing log events with the record_transformer plugin
153(3)
Filter parser vs. record transformer
156(1)
6.4 Demonstrating change impact with stdout in action
157(1)
A solution demonstrating change impact with stdout in action
157(1)
6.5 Extract to set key values
157(2)
6.6 Deriving new data values with the record_transformer
159(1)
Putting the incorporation of calculations into a log event transformation into action
160(1)
6.7 Generating simple Fluentd metrics
160(5)
Putting log event counting into action
163(2)
Part 3 Beyond the basics
165(92)
7 Performance and scaling
167(30)
7.1 Threading and processes to scale with workers
168(5)
Seeing workers in action
169(2)
Worker constraints
171(1)
Controlling output plugin threads
172(1)
Memory management optimization
172(1)
7.2 Scaling and moving workloads
173(12)
Fan-in/log aggregation and consolidation
173(6)
Fan-out and workload distribution
179(5)
High availability
184(1)
Putting a high-availability comparison into action
185(1)
7.3 Fluentd scaling in containers vs. native and virtual environments
185(5)
Kubernetes worker node configuration
186(1)
Per-cluster configuration
186(1)
Container as visualization
187(1)
Sidecar pattern
188(1)
Options comparison
189(1)
7.4 Securing traffic between Fluentd nodes
190(2)
TLS configuration
190(1)
TLS not just for encryption
191(1)
Certificate and private key storage
191(1)
Security is more than certificates
192(1)
7.5 Credentials management
192(5)
Simple credentials use case
193(2)
Putting certification into action
195(2)
8 Driving logs with Docker and Kubernetes
197(34)
8.1 Fluentd out of the box from Docker Hub
198(2)
Official Docker images
198(1)
Docker log drivers
199(1)
Getting set up for Docker log drivers
199(1)
8.2 Using Docker log drivers
200(7)
Docker drivers via the command line
200(1)
A quick check of network connections
201(1)
Running Docker command line
202(2)
Switching to driver configuration through a configuration file
204(3)
8.3 Kubernetes components logging and the use of Fluentd
207(2)
Kubernetes components and structured logging
208(1)
Kubernetes default log retention and log rotation
208(1)
Kubectl with logging
209(1)
8.4 Demonstrating logging with Kubernetes
209(10)
Kubernetes setup
210(2)
Creating logs to capture
212(3)
Understanding how Fluentd DaemonSets are put together
215(4)
8.5 Getting a peek at host logs
219(3)
8.6 Configuring a Kubernetes logging DaemonSet
222(6)
Getting the Fluentd configuration ready to be used
222(2)
Creating our Kubernetes deployment configuration
224(2)
Putting the implementation of a Fluentd for Kubernetes into action
226(1)
Deploying to minikube
227(1)
Tidying up
228(1)
8.7 Kubernetes configuration in action
228(1)
Answer
228(1)
8.8 More Kubernetes monitoring and logging to watch for
228(3)
Node monitoring
229(1)
Termination messages
229(2)
9 Creating custom plugins
231(26)
9.1 Plugin source code
232(1)
9.2 What is Redis, and why build a plugin with the Redis list capability?
232(1)
Redis list over RedisTimeSeries
233(1)
9.3 Illustrating our objective using Redis CLI
233(1)
9.4 Preparing for development
234(1)
9.5 Plugin frameworks
235(3)
Creating the skeleton plugin
235(2)
Plugin life cycle
237(1)
9.6 Implementing the plugin core
238(6)
How configuration attributes work
238(2)
Starting up and shutting down
240(1)
Getting the plugin to work with our Fluentd installation
241(1)
Putting additional configuration validation into action
242(1)
Implementing the Redis output logic
243(1)
Putting the testing of synchronous output into action
244(1)
9.7 Implementing the Redis input plugin
244(3)
Testing input and output plugin execution
246(1)
9.8 Extending output with buffering
247(3)
Improving our scenario by putting maintainability into action
250(1)
9.9 Unit testing
250(1)
9.10 Putting the development of unit tests into action
251(1)
Answer
251(1)
9.11 Package and deployment
252(2)
Documentation
252(1)
Complete metadata aka manifest
253(1)
Building the gem package
253(1)
Rerun without the plugin paths
253(1)
9.12 Extending to be an enterprise-class solution
254(3)
Part 4 Good logging practices and frameworks to maximize log value
257(42)
10 Logging best practices
259(22)
10.1 Audit events vs. log events
260(1)
10.2 Log levels and severities
260(4)
Trace
261(1)
Debug
261(1)
Info(rmation)
262(1)
Warn(ing)
262(1)
Error
263(1)
Fatal
263(1)
Extending or creating your own log levels
263(1)
10.3 Clear language
264(1)
10.4 Human and machine-readable
265(1)
10.5 Context is key
265(4)
Context: What
266(1)
Context: When
266(1)
Context: where
266(1)
Context: Why
267(1)
Context: Who
268(1)
A Practical checklist for capturing context
268(1)
10.6 Error codes
269(2)
Using standard errors
271(1)
Codes can be for more than errors
271(1)
10.7 Too little logging or too much?
271(4)
What qualifies as sensitive?
272(2)
GDPR is only the start
274(1)
10.8 Log structure and format
275(2)
Putting making log entries ready for application shipping into action
276(1)
10.9 Use frameworks if you can
277(1)
10.10 Development practices
277(4)
Rethrowing exceptions
278(1)
Using standard exceptions and error structures
278(1)
String construction as a reason not to log
279(2)
11 Logging frameworks
281(18)
11.1 Value of logging frameworks
282(1)
11.2 Typical structure of a logging framework
283(2)
Logger context
283(1)
Appender
284(1)
Logger
284(1)
Filter
285(1)
Formatter
285(1)
Configuration
285(1)
Logger config
285(1)
11.3 Appender structures
285(1)
11.4 Logging framework landscape
286(1)
11.5 Choosing a framework
287(1)
Putting optimizing application logging into action
288(1)
11.6 Fluentd's own logging and appenders
288(2)
11.7 Illustrations of an application logging directly to Fluentd
290(9)
Python with logging framework: Using the Fluentd library
290(3)
Invoking Fluentd appender directly
293(1)
Illustration with only Python's logging
294(1)
Illustration without Python's logging or Fluentd library
295(1)
Porting the Fluentd calls to another language into action
296(1)
Using generic appenders: The takeaways
297(2)
Appendix A Installation of additional tools and services 299(16)
Appendix B Processing times and dates, regular expressions, and other configuration values 315(6)
Appendix C Plugins summary 321(6)
Appendix D Real-world use case 327(10)
Appendix E Useful resources 337(16)
Index 353
Phil Wilkins has spent over 25 years in the software industry. He works as a Technology Evangelist for Capgemini, specializing in cloud integration, APIs, and related technologies. Phil is TOGAF certified, and recognized by Oracle as an Ace Director (independent technology advocate) for his contributions to the integration and PaaS community.
Lisainfo e-raamatute kohta Lisainfo e-raamatute kohta
Püsilink: https://www.kriso.ee/db/97816383556706e.html
Märksõnad: