1 Introduction |
|
1 | (26) |
|
1.1 The Monte Carlo Method: A Brief History |
|
|
1 | (3) |
|
1.2 The Need for Monte Carlo |
|
|
4 | (5) |
|
1.2.1 Numerical Integration |
|
|
4 | (2) |
|
1.2.2 Importance Sampling |
|
|
6 | (2) |
|
|
8 | (1) |
|
1.2.4 Inverse Monte Carlo |
|
|
8 | (1) |
|
1.3 Random Number Generation |
|
|
9 | (2) |
|
1.3.1 Random, Pseudo-Random, Quasi-Random |
|
|
9 | (2) |
|
1.4 Pseudo-Random Number Generators |
|
|
11 | (5) |
|
1.4.1 Nonlinear Recursions |
|
|
11 | (1) |
|
1.4.2 Chaotic Pseudo-Random Number Generators |
|
|
12 | (2) |
|
1.4.3 The Middle-Square Generator |
|
|
14 | (1) |
|
1.4.4 Linear Congruential Generators |
|
|
14 | (2) |
|
1.5 Random Sampling Methods |
|
|
16 | (3) |
|
|
17 | (1) |
|
1.5.2 Accept/Reject Methods |
|
|
17 | (1) |
|
1.5.3 Markov Chain Monte Carlo (MCMC) |
|
|
18 | (1) |
|
1.5.4 Importance Sampling |
|
|
18 | (1) |
|
|
18 | (1) |
|
1.6 Goal and Organization of This Book |
|
|
19 | (2) |
|
1.6.1 Motivation and Goals |
|
|
19 | (1) |
|
1.6.2 Organization of the Book |
|
|
20 | (1) |
|
|
21 | (6) |
2 Direct Methods |
|
27 | (38) |
|
|
27 | (1) |
|
|
28 | (1) |
|
2.2.1 Vectors, Points, and Intervals |
|
|
28 | (1) |
|
2.2.2 Random Variables, Distributions, and Densities |
|
|
29 | (1) |
|
|
29 | (1) |
|
2.3 Transformations of Random Variables |
|
|
29 | (12) |
|
2.3.1 One-to-One Transformations |
|
|
30 | (3) |
|
2.3.2 Many-to-One Transformations |
|
|
33 | (4) |
|
2.3.3 Deconvolution Method |
|
|
37 | (1) |
|
|
38 | (1) |
|
2.3.5 Continuous Mixtures: Marginalization |
|
|
39 | (1) |
|
|
39 | (2) |
|
2.4 Universal Direct Methods |
|
|
41 | (12) |
|
|
41 | (4) |
|
2.4.2 Vertical Density Representation (VDR) |
|
|
45 | (4) |
|
2.4.3 The Fundamental Theorem of Simulation |
|
|
49 | (1) |
|
2.4.4 Inverse-of-Density Method |
|
|
50 | (3) |
|
|
53 | (3) |
|
|
53 | (2) |
|
|
55 | (1) |
|
|
56 | (4) |
|
2.6.1 Multiplication of Independent Uniform Random Variates |
|
|
56 | (2) |
|
2.6.2 Sum of Independent Uniform Random Variates |
|
|
58 | (1) |
|
2.6.3 Polynomial Densities with Non-negative Coefficients |
|
|
59 | (1) |
|
2.6.4 Polynomial Densities with One or More Negative Constants |
|
|
60 | (1) |
|
|
60 | (2) |
|
|
62 | (3) |
3 Accept-Reject Methods |
|
65 | (50) |
|
|
65 | (1) |
|
|
66 | (8) |
|
|
69 | (1) |
|
3.2.2 Distribution of the Rejected Samples |
|
|
69 | (1) |
|
3.2.3 Distribution of the Accepted and Rejected Samples with Generic L > 0 |
|
|
70 | (1) |
|
3.2.4 Different Application Scenarios |
|
|
70 | (1) |
|
3.2.5 Butcher's Version of the Rejection Sampler |
|
|
71 | (1) |
|
3.2.6 Vaduva's Modification of the Butcher's Method |
|
|
72 | (1) |
|
|
73 | (1) |
|
|
74 | (6) |
|
3.3.1 Further Considerations About the Acceptance Rate |
|
|
75 | (3) |
|
|
78 | (1) |
|
3.3.3 Sibuya's Modified Rejection Method |
|
|
79 | (1) |
|
3.4 Band Rejection Method |
|
|
80 | (6) |
|
|
80 | (2) |
|
3.4.2 Generalized Band Rejection Algorithm |
|
|
82 | (3) |
|
3.4.3 Payne-Dagpunar's Band Rejection |
|
|
85 | (1) |
|
3.5 Acceptance-Complement Method |
|
|
86 | (3) |
|
3.6 RS with Stepwise Proposals |
|
|
89 | (5) |
|
|
90 | (2) |
|
3.6.2 Inversion-Rejection Method |
|
|
92 | (2) |
|
3.7 Transformed Rejection Method |
|
|
94 | (5) |
|
3.7.1 Transformed Rejection and IoD Method |
|
|
97 | (2) |
|
|
99 | (4) |
|
3.8.1 RS for Generating Order Statistics |
|
|
99 | (1) |
|
3.8.2 Mixtures with Negative Coefficients |
|
|
100 | (2) |
|
3.8.3 Pdfs Expressed as Sequences of Functions |
|
|
102 | (1) |
|
3.9 Monte Carlo Estimation via RS |
|
|
103 | (7) |
|
3.9.1 Recycling Rejected Samples |
|
|
105 | (1) |
|
3.9.2 RS with a Generic Constant L > 0 |
|
|
106 | (4) |
|
|
110 | (1) |
|
|
111 | (4) |
4 Adaptive Rejection Sampling Methods |
|
115 | (44) |
|
|
115 | (1) |
|
4.2 Generic Structure of an Adaptive Rejection Sampler |
|
|
116 | (3) |
|
|
117 | (1) |
|
4.2.2 Generic Adaptive Algorithm |
|
|
117 | (2) |
|
4.3 Constructions of the Proposal Densities |
|
|
119 | (28) |
|
4.3.1 Standard Adaptive Rejection Sampling |
|
|
119 | (6) |
|
4.3.2 Derivative-Free Constructions for Log-Concave pdfs |
|
|
125 | (2) |
|
|
127 | (1) |
|
4.3.4 Transformed Density Rejection |
|
|
128 | (4) |
|
|
132 | (3) |
|
4.3.6 Generalized Adaptive Rejection Sampling |
|
|
135 | (12) |
|
4.4 Performance and Computational Cost of the ARS Schemes |
|
|
147 | (1) |
|
|
147 | (1) |
|
4.4.2 Probability of Adding a New Support Point |
|
|
148 | (1) |
|
4.5 Variants of the Adaptive Structure in the ARS Scheme |
|
|
148 | (5) |
|
4.5.1 Deterministic Test for Adding New Support Points |
|
|
149 | (2) |
|
4.5.2 An Adaptive Rejection Sampler with Fixed Number of Support Points |
|
|
151 | (2) |
|
4.6 Combining ARS and MCMC |
|
|
153 | (2) |
|
4.6.1 Adaptive Rejection Metropolis Sampling |
|
|
153 | (1) |
|
4.6.2 A Procedure to Build Proposal pdfs for the ARMS Algorithm |
|
|
154 | (1) |
|
|
155 | (1) |
|
|
156 | (3) |
5 Ratio of Uniforms |
|
159 | (38) |
|
|
159 | (2) |
|
5.1.1 A Remark on Inverse Densities |
|
|
160 | (1) |
|
5.2 Standard Ratio of Uniforms Method |
|
|
161 | (6) |
|
5.2.1 Some Basic Considerations |
|
|
163 | (1) |
|
|
164 | (3) |
|
5.3 Envelope Polygons and Adaptive RoU |
|
|
167 | (2) |
|
5.4 Generalized Ratio of Uniforms Method |
|
|
169 | (2) |
|
5.5 Properties of Generalized RoU Samplers |
|
|
171 | (5) |
|
|
171 | (1) |
|
5.5.2 How to Guarantee that Ag is Bounded |
|
|
171 | (3) |
|
|
174 | (2) |
|
5.6 Connections Between GRoU and Other Classical Techniques |
|
|
176 | (7) |
|
5.6.1 Extended Inverse-of-Density Method |
|
|
176 | (4) |
|
5.6.2 GRoU Sampling and the Transformed Rejection Method |
|
|
180 | (3) |
|
5.6.3 Role of the Constant CA |
|
|
183 | (1) |
|
5.7 How Does GRoU Work for Generic pdfs? |
|
|
183 | (6) |
|
5.7.1 IoD for Arbitrary pdfs |
|
|
184 | (1) |
|
5.7.2 GRoU for pdfs with a Single Mode at x = 0 |
|
|
185 | (1) |
|
5.7.3 GRoU for pdfs with a Single Mode at x not = to 0 |
|
|
186 | (1) |
|
5.7.4 GRoU for Arbitrary pdfs |
|
|
187 | (1) |
|
|
188 | (1) |
|
5.8 Rectangular Region Ag |
|
|
189 | (2) |
|
5.8.1 Yet Another Connection Between IoD and GRoU |
|
|
190 | (1) |
|
5.9 Relaxing Assumptions: GRoU with Decreasing g(u) |
|
|
191 | (2) |
|
5.9.1 General Expression of a r.v. Transformation |
|
|
192 | (1) |
|
5.10 Another View of GRoU |
|
|
193 | (1) |
|
|
194 | (3) |
|
|
195 | (2) |
6 Independent Sampling for Multivariate Densities |
|
197 | (52) |
|
|
197 | (1) |
|
|
198 | (1) |
|
|
199 | (8) |
|
6.3.1 Chain Rule Decomposition |
|
|
199 | (1) |
|
6.3.2 Dependence Generation |
|
|
200 | (3) |
|
|
203 | (2) |
|
6.3.4 RoU for Multivariate Densities |
|
|
205 | (2) |
|
6.4 Elliptically Contoured Distributions |
|
|
207 | (3) |
|
|
208 | (2) |
|
6.5 Vertical Density Representation |
|
|
210 | (2) |
|
6.5.1 Inverse-of-Density Method |
|
|
212 | (1) |
|
6.6 Uniform Distributions in Dimension n |
|
|
212 | (4) |
|
6.6.1 Points Uniformly Distributed in a Simplex |
|
|
213 | (1) |
|
6.6.2 Sampling Uniformly Within a Hypersphere |
|
|
214 | (2) |
|
6.6.3 Points Uniformly Distributed Within a Hyperellipsoid |
|
|
216 | (1) |
|
6.7 Transformations of a Random Variable |
|
|
216 | (8) |
|
6.7.1 Many-to-Few Transformations (m > n) |
|
|
217 | (1) |
|
6.7.2 Few-to-Many Transformations: Singular Distributions (m < n) |
|
|
218 | (3) |
|
6.7.3 Sampling a Uniform Distribution on a Differentiable Manifold |
|
|
221 | (3) |
|
6.8 Sampling Techniques for Specific Distributions |
|
|
224 | (8) |
|
6.8.1 Multivariate Gaussian Distribution |
|
|
224 | (1) |
|
6.8.2 Multivariate Student's t-Distribution |
|
|
225 | (1) |
|
6.8.3 Wishart Distribution |
|
|
225 | (1) |
|
6.8.4 Inverse Wishart Distribution |
|
|
226 | (1) |
|
6.8.5 Multivariate Gamma Samples |
|
|
227 | (1) |
|
6.8.6 Dirichlet Distribution |
|
|
227 | (1) |
|
6.8.7 Cook-Johnson's Family |
|
|
228 | (1) |
|
6.8.8 Some Relevant Bivariate Distributions |
|
|
229 | (3) |
|
6.9 Generation of Stochastic Processes |
|
|
232 | (13) |
|
6.9.1 Markov Jump Processes |
|
|
232 | (2) |
|
|
234 | (3) |
|
|
237 | (1) |
|
|
238 | (2) |
|
|
240 | (3) |
|
6.9.6 Dirichlet Processes: "Rich Get Richer" |
|
|
243 | (2) |
|
|
245 | (1) |
|
|
246 | (3) |
7 Asymptotically Independent Samplers |
|
249 | (18) |
|
|
249 | (2) |
|
7.2 Metropolis-Hastings (MH) Methods |
|
|
251 | (3) |
|
|
251 | (1) |
|
7.2.2 Invariant Distribution of the MH Algorithm |
|
|
252 | (1) |
|
7.2.3 Acceptance Rate in MH-Type Methods |
|
|
253 | (1) |
|
7.3 Independent Generalized MH Methods with Multiple Candidates |
|
|
254 | (3) |
|
7.3.1 Independent Multiple Try Metropolis Algorithms |
|
|
254 | (2) |
|
7.3.2 Ensemble MCMC Method |
|
|
256 | (1) |
|
7.4 Independent Doubly Adaptive Rejection Metropolis Sampling |
|
|
257 | (8) |
|
7.4.1 Adaptive Rejection Sampling (ARS) |
|
|
258 | (1) |
|
7.4.2 Adaptive Rejection Metropolis Sampling |
|
|
259 | (1) |
|
7.4.3 Structural Limitations of ARMS |
|
|
260 | (1) |
|
|
261 | (2) |
|
7.4.5 Convergence of the Chain and Computational Cost |
|
|
263 | (1) |
|
7.4.6 Examples of Proposal Constructions for IA2RMS |
|
|
263 | (2) |
|
|
265 | (1) |
|
|
265 | (2) |
8 Summary and Outlook |
|
267 | (4) |
|
|
270 | (1) |
A Acronyms and Abbreviations |
|
271 | (2) |
B Notation |
|
273 | (2) |
|
B.1 Vectors, Points, and Intervals |
|
|
273 | (1) |
|
B.2 Random Variables, Distributions and Densities |
|
|
273 | (1) |
|
|
274 | (1) |
|
B.4 Summary of Main Notation |
|
|
274 | (1) |
C Jones' RoU Generalization |
|
275 | (4) |
|
C.1 Possible Choices of t(v, u) |
|
|
277 | (1) |
|
|
278 | (1) |
D Polar Transformation |
|
279 | |