Preface |
|
xi | |
Acknowledgments |
|
xv | |
|
List of Symbols and Notation |
|
|
xvii | |
About the Companion Website |
|
xxi | |
|
1 Introduction and Review |
|
|
1 | (39) |
|
1.1 Deterministic and Stochastic Models |
|
|
1 | (5) |
|
1.2 What is a Stochastic Process? |
|
|
6 | (3) |
|
1.3 Monte Carlo Simulation |
|
|
9 | (1) |
|
1.4 Conditional Probability |
|
|
10 | (8) |
|
1.5 Conditional Expectation |
|
|
18 | (22) |
|
|
34 | (6) |
|
2 Markov Chains: First Steps |
|
|
40 | (36) |
|
|
40 | (2) |
|
2.2 Markov Chain Cornucopia |
|
|
42 | (10) |
|
|
52 | (7) |
|
2.4 Long-Term Behavior---the Numerical Evidence |
|
|
59 | (6) |
|
|
65 | (3) |
|
2.6 Mathematical Induction* |
|
|
68 | (8) |
|
|
70 | (6) |
|
3 Markov Chains for the Long Term |
|
|
76 | (82) |
|
3.1 Limiting Distribution |
|
|
76 | (4) |
|
3.2 Stationary Distribution |
|
|
80 | (14) |
|
3.3 Can you Find the Way to State a? |
|
|
94 | (9) |
|
3.4 Irreducible Markov Chains |
|
|
103 | (3) |
|
|
106 | (3) |
|
3.6 Ergodic Markov Chains |
|
|
109 | (5) |
|
|
114 | (5) |
|
|
119 | (14) |
|
3.9 Regeneration and the Strong Markov Property* |
|
|
133 | (2) |
|
3.10 Proofs of Limit Theorems* |
|
|
135 | (23) |
|
|
144 | (14) |
|
|
158 | (23) |
|
|
158 | (2) |
|
|
160 | (4) |
|
4.3 Probability Generating Functions |
|
|
164 | (4) |
|
4.4 Extinction is Forever |
|
|
168 | (13) |
|
|
175 | (6) |
|
5 Markov Chain Monte Carlo |
|
|
181 | (42) |
|
|
181 | (6) |
|
5.2 Metropolis--Hastings Algorithm |
|
|
187 | (10) |
|
|
197 | (8) |
|
|
205 | (5) |
|
5.5 Rate of Convergence: the Eigenvalue Connection* |
|
|
210 | (2) |
|
5.6 Card Shuffling and Total Variation Distance* |
|
|
212 | (11) |
|
|
219 | (4) |
|
|
223 | (42) |
|
|
223 | (4) |
|
6.2 Arrival, Interarrival Times |
|
|
227 | (7) |
|
6.3 Infinitesimal Probabilities |
|
|
234 | (4) |
|
6.4 Thinning, Superposition |
|
|
238 | (5) |
|
|
243 | (6) |
|
6.6 Spatial Poisson Process |
|
|
249 | (4) |
|
6.7 Nonhomogeneous Poisson Process |
|
|
253 | (2) |
|
|
255 | (10) |
|
|
258 | (7) |
|
7 Continuous-Time Markov Chains |
|
|
265 | (55) |
|
|
265 | (5) |
|
7.2 Alarm Clocks and Transition Rates |
|
|
270 | (3) |
|
7.3 Infinitesimal Generator |
|
|
273 | (10) |
|
|
283 | (11) |
|
|
294 | (7) |
|
|
301 | (5) |
|
7.7 Poisson Subordination |
|
|
306 | (14) |
|
|
313 | (7) |
|
|
320 | (52) |
|
|
320 | (6) |
|
8.2 Brownian Motion and Random Walk |
|
|
326 | (4) |
|
|
330 | (4) |
|
8.4 Transformations and Properties |
|
|
334 | (11) |
|
8.5 Variations and Applications |
|
|
345 | (11) |
|
|
356 | (16) |
|
|
366 | (6) |
|
9 A Gentle Introduction to Stochastic Calculus* |
|
|
372 | (28) |
|
|
372 | (6) |
|
|
378 | (7) |
|
9.3 Stochastic Differential Equations |
|
|
385 | (15) |
|
|
397 | (3) |
|
|
400 | (21) |
|
|
421 | (22) |
|
B.1 Discrete Random Variables |
|
|
422 | (2) |
|
|
424 | (2) |
|
B.3 Continuous Random Variables |
|
|
426 | (2) |
|
B.4 Common Probability Distributions |
|
|
428 | (11) |
|
|
439 | (1) |
|
B.6 Moment-Generating Functions |
|
|
440 | (3) |
|
C Summary of Common Probability Distributions |
|
|
443 | (2) |
|
|
445 | (10) |
|
|
445 | (2) |
|
|
447 | (1) |
|
D.3 Matrix Multiplication |
|
|
448 | (1) |
|
D.4 Diagonal, Identity Matrix, Polynomials |
|
|
448 | (1) |
|
|
449 | (1) |
|
|
449 | (1) |
|
|
449 | (1) |
|
D.8 Linear Independence and Span |
|
|
450 | (1) |
|
|
451 | (1) |
|
|
451 | (1) |
|
|
452 | (1) |
|
D.12 Eigenvalue, Eigenvector |
|
|
452 | (1) |
|
|
453 | (2) |
Answers to Selected Odd-Numbered Exercises |
|
455 | (15) |
References |
|
470 | (5) |
Index |
|
475 | |