Preface |
|
xi | |
Nomenclature |
|
xiii | |
|
|
1 | (24) |
|
|
2 | (1) |
|
1.2 Events and Probabilities |
|
|
3 | (8) |
|
|
3 | (2) |
|
1.2.2 Assigning Probabilities |
|
|
5 | (1) |
|
1.2.3 Joint and Conditional Probabilities |
|
|
5 | (3) |
|
|
8 | (3) |
|
|
11 | (1) |
|
1.4 Probability Densities |
|
|
12 | (3) |
|
|
15 | (4) |
|
1.5.1 Indicator Functions |
|
|
16 | (1) |
|
|
17 | (2) |
|
1.6 Probabilistic Inference |
|
|
19 | (4) |
|
|
23 | (2) |
|
2 Survey of Distributions |
|
|
25 | (20) |
|
2.1 Discrete Distributions |
|
|
26 | (4) |
|
2.1.1 Binomial Distribution |
|
|
26 | (1) |
|
2.1.2 Hypergeometric Distribution |
|
|
27 | (1) |
|
2.1.3 Poisson Distribution |
|
|
28 | (2) |
|
2.2 Continuous Distributions |
|
|
30 | (5) |
|
2.2.1 Normal Distribution |
|
|
30 | (1) |
|
|
31 | (3) |
|
|
34 | (1) |
|
2.2.4 Cauchy Distribution |
|
|
35 | (1) |
|
2.3 Multivariate Distributions |
|
|
35 | (3) |
|
2.3.1 Multinomial Distribution |
|
|
35 | (2) |
|
2.3.2 Multivariate Normal Distribution |
|
|
37 | (1) |
|
2.3.3 Dirichlet Distribution |
|
|
37 | (1) |
|
|
38 | (3) |
|
|
41 | (2) |
|
|
43 | (2) |
|
|
45 | (14) |
|
|
46 | (3) |
|
3.1.1 Estimating Expectations |
|
|
46 | (1) |
|
3.1.2 Monte Carlo Integration |
|
|
47 | (2) |
|
3.2 Uniform Random Deviates |
|
|
49 | (2) |
|
3.3 Non-Uniform Random Deviates |
|
|
51 | (8) |
|
3.3.1 Transformation Method |
|
|
51 | (3) |
|
|
54 | (2) |
|
3.3.3 Multivariate Deviates |
|
|
56 | (3) |
|
4 Discrete Random Variables |
|
|
59 | (15) |
|
|
60 | (1) |
|
4.2 Binomial Distribution |
|
|
61 | (5) |
|
4.2.1 Statistical Properties of the Binomial Distribution |
|
|
62 | (2) |
|
|
64 | (2) |
|
4.3 Beyond the Binomial Distribution |
|
|
66 | (8) |
|
|
66 | (1) |
|
4.3.2 Poisson Distribution |
|
|
67 | (1) |
|
4.3.3 Multinomial Distribution |
|
|
68 | (6) |
|
5 The Normal Distribution |
|
|
74 | (36) |
|
|
75 | (1) |
|
|
76 | (5) |
|
5.2.1 Integrating Gaussians |
|
|
76 | (2) |
|
5.2.2 Moments and Cumulants |
|
|
78 | (3) |
|
5.3 Central Limit Theorem |
|
|
81 | (9) |
|
5.4 Cumulative Distribution Function of a Normal Distribution |
|
|
90 | (2) |
|
|
92 | (4) |
|
5.6 Multivariate Normal Distributions |
|
|
96 | (7) |
|
|
103 | (4) |
|
5.B Characteristic Function for the Cauchy Distribution |
|
|
107 | (3) |
|
6 Handling Experimental Data |
|
|
110 | (22) |
|
6.1 Estimating the Error in the Mean |
|
|
111 | (4) |
|
6.1.1 Computing Means with Errors |
|
|
114 | (1) |
|
|
114 | (1) |
|
|
115 | (1) |
|
|
116 | (7) |
|
6.4 Maximum Likelihood Estimate |
|
|
123 | (3) |
|
6.A Deriving the T-Distribution |
|
|
126 | (6) |
|
7 Mathematics of Random Variables |
|
|
132 | (55) |
|
|
133 | (9) |
|
7.1.1 Laws of Large Numbers |
|
|
138 | (1) |
|
|
139 | (3) |
|
|
142 | (24) |
|
7.2.1 Cauchy-Schwarz Inequality |
|
|
142 | (1) |
|
7.2.2 Markov's Inequality |
|
|
143 | (6) |
|
7.2.3 Chebyshev's Inequality |
|
|
149 | (3) |
|
7.2.4 Tail Bounds and Concentration Theorems |
|
|
152 | (5) |
|
7.2.5 Chernoff Bounds for Independent Bernoulli Trials |
|
|
157 | (2) |
|
7.2.6 Jensen's Inequality |
|
|
159 | (6) |
|
|
165 | (1) |
|
7.3 Comparing Distributions |
|
|
166 | (11) |
|
7.3.1 Kullback-Leibler Divergence |
|
|
166 | (4) |
|
7.3.2 Wasserstein Distance |
|
|
170 | (7) |
|
|
177 | (2) |
|
7.B Hoeffding's Bound for Negatively Correlated Random Boolean Variables |
|
|
179 | (3) |
|
7.C Constrained Optimisation |
|
|
182 | (5) |
|
|
187 | (71) |
|
|
188 | (9) |
|
8.2 Performing Bayesian Inference |
|
|
197 | (17) |
|
|
198 | (7) |
|
8.2.2 Uninformative Priors |
|
|
205 | (6) |
|
|
211 | (3) |
|
8.3 Bayes with Complex Likelihoods |
|
|
214 | (13) |
|
8.3.1 Hierarchical Models |
|
|
214 | (1) |
|
|
215 | (5) |
|
8.3.3 Variational Approximation |
|
|
220 | (7) |
|
8.4 Latent Variable Models |
|
|
227 | (16) |
|
8.4.1 Hidden Markov Models |
|
|
235 | (5) |
|
8.4.2 Variational Auto-Encoders |
|
|
240 | (3) |
|
|
243 | (13) |
|
8.5.1 Naive Bayes Classifier |
|
|
244 | (1) |
|
|
245 | (11) |
|
|
256 | (2) |
|
|
258 | (35) |
|
|
259 | (8) |
|
|
259 | (6) |
|
9.1.2 Properties of Entropy |
|
|
265 | (2) |
|
|
267 | (9) |
|
|
267 | (1) |
|
|
267 | (5) |
|
9.2.3 The Second Law of Thermodynamics |
|
|
272 | (4) |
|
9.3 Beyond Information Theory |
|
|
276 | (14) |
|
9.3.1 Kolmogorov Complexity |
|
|
276 | (1) |
|
9.3.2 Minimum Description Length |
|
|
277 | (8) |
|
|
285 | (5) |
|
|
290 | (3) |
|
|
293 | (15) |
|
|
294 | (1) |
|
|
295 | (2) |
|
|
297 | (2) |
|
|
299 | (3) |
|
10.5 Self-Organised Criticality |
|
|
302 | (2) |
|
|
304 | (4) |
|
|
308 | (41) |
|
|
309 | (8) |
|
11.1.1 Markov Chains and Stochastic Matrices |
|
|
310 | (3) |
|
11.1.2 Properties of Stochastic Matrices |
|
|
313 | (3) |
|
11.1.3 Ergodic Stochastic Matrices |
|
|
316 | (1) |
|
11.2 Markov Chain Monte Carlo |
|
|
317 | (13) |
|
|
318 | (5) |
|
|
323 | (3) |
|
11.2.3 Convergence of MCMC |
|
|
326 | (3) |
|
11.2.4 Hybrid Monte Carlo |
|
|
329 | (1) |
|
11.3 Kalman and Particle Filtering |
|
|
330 | (17) |
|
|
332 | (6) |
|
11.3.2 Particle Filtering |
|
|
338 | (6) |
|
11.3.3 Approximate Bayesian Computation |
|
|
344 | (3) |
|
11.A Eigenvalues and Eigenvectors of General Square Matrices |
|
|
347 | (2) |
|
|
349 | (42) |
|
12.1 Stochastic Processes |
|
|
350 | (9) |
|
12.1.1 What Are Stochastic Processes? |
|
|
350 | (1) |
|
12.1.2 Gaussian Processes |
|
|
351 | (5) |
|
|
356 | (3) |
|
|
359 | (21) |
|
|
359 | (2) |
|
12.2.2 Stochastic Differential Equations |
|
|
361 | (9) |
|
12.2.3 Fokker-Planck Equation |
|
|
370 | (4) |
|
12.2.4 Stationary Distribution of Stochastic Processes |
|
|
374 | (3) |
|
|
377 | (3) |
|
|
380 | (11) |
|
|
380 | (3) |
|
12.3.2 Poisson Processes in One Dimension |
|
|
383 | (2) |
|
12.3.3 Chemical Reactions |
|
|
385 | (6) |
|
|
391 | (54) |
|
A.1 Answers to Chapter 1. Introduction |
|
|
392 | (4) |
|
A.2 Answers to Chapter 2. Survey of Distributions |
|
|
396 | (3) |
|
A.3 Answers to Chapter 3. Monte Carlo |
|
|
399 | (2) |
|
A.4 Answers to Chapter 4. Discrete Random Variables |
|
|
401 | (6) |
|
A.5 Answers to Chapter 5. The Normal Distribution |
|
|
407 | (4) |
|
A.6 Answers to Chapter 6. Handling Experimental Data |
|
|
411 | (3) |
|
A.7 Answers to Chapter 7. Mathematics of Random Variables |
|
|
414 | (7) |
|
A.8 Answers to Chapter 8. Bayes |
|
|
421 | (6) |
|
A.9 Answers to Chapter 9. Entropy |
|
|
427 | (3) |
|
A.10 Answers to Chapter 10. Collective Behaviour |
|
|
430 | (2) |
|
A.11 Answers to Chapter 11. Markov Chains |
|
|
432 | (9) |
|
A.12 Answers to Chapter 12. Stochastic Processes |
|
|
441 | (4) |
|
B Probability Distributions |
|
|
445 | (4) |
|
Table B1 Discrete Univariate Distributions |
|
|
446 | (1) |
|
Table B2 Continuous Univariate Distributions |
|
|
447 | (1) |
|
Table B3 Continuous and Discrete Multivariate Distributions |
|
|
448 | (1) |
Bibliography |
|
449 | (5) |
Index |
|
454 | |