Preface |
|
xi | |
|
|
1 | (21) |
|
1.1 The Laws of Probability |
|
|
1 | (4) |
|
1.2 Probability Distributions |
|
|
5 | (4) |
|
1.2.1 Discrete and Continuous Probability Distributions |
|
|
5 | (3) |
|
1.2.2 Cumulative Probability Distribution Function |
|
|
8 | (1) |
|
1.2.3 Change of Variables |
|
|
8 | (1) |
|
1.3 Characterizations of Probability Distributions |
|
|
9 | (7) |
|
1.3.1 Medians, Modes, and Full Width at Half Maximum |
|
|
9 | (1) |
|
1.3.2 Moments, Means, and Variances |
|
|
10 | (4) |
|
1.3.3 Moment Generating Function and the Characteristic Function |
|
|
14 | (2) |
|
1.4 Multivariate Probability Distributions |
|
|
16 | (6) |
|
1.4.1 Distributions with Two Independent Variables |
|
|
16 | (1) |
|
|
17 | (2) |
|
1.4.3 Distributions with Many Independent Variables |
|
|
19 | (3) |
|
2 Some Useful Probability Distribution Functions |
|
|
22 | (28) |
|
2.1 Combinations and Permutations |
|
|
22 | (2) |
|
2.2 Binomial Distribution |
|
|
24 | (3) |
|
|
27 | (4) |
|
2.4 Gaussian or Normal Distribution |
|
|
31 | (6) |
|
2.4.1 Derivation of the Gaussian Distribution---Central Limit Theorem |
|
|
31 | (3) |
|
2.4.2 Summary and Comments on the Central Limit Theorem |
|
|
34 | (2) |
|
2.4.3 Mean, Moments, and Variance of the Gaussian Distribution |
|
|
36 | (1) |
|
2.5 Multivariate Gaussian Distribution |
|
|
37 | (4) |
|
|
41 | (6) |
|
2.6.1 Derivation of the Χ2 Distribution |
|
|
41 | (3) |
|
2.6.2 Mean, Mode, and Variance of the Χ2 Distribution |
|
|
44 | (1) |
|
2.6.3 Χ2 Distribution in the Limit of Large n |
|
|
45 | (1) |
|
|
46 | (1) |
|
2.6.5 Χ2 for Correlated Variables |
|
|
46 | (1) |
|
|
47 | (3) |
|
3 Random Numbers and Monte Carlo Methods |
|
|
50 | (31) |
|
|
50 | (1) |
|
3.2 Nonuniform Random Deviates |
|
|
51 | (8) |
|
3.2.1 Inverse Cumulative Distribution Function Method |
|
|
52 | (1) |
|
3.2.2 Multidimensional Deviates |
|
|
53 | (1) |
|
3.2.3 Box-Muller Method for Generating Gaussian Deviates |
|
|
53 | (1) |
|
3.2.4 Acceptance-Rejection Algorithm |
|
|
54 | (3) |
|
3.2.5 Ratio of Uniforms Method |
|
|
57 | (2) |
|
3.2.6 Generating Random Deviates from More Complicated Probability Distributions |
|
|
59 | (1) |
|
3.3 Monte Carlo Integration |
|
|
59 | (4) |
|
|
63 | (8) |
|
3.4.1 Stationary, Finite Markov Chains |
|
|
63 | (2) |
|
3.4.2 Invariant Probability Distributions |
|
|
65 | (3) |
|
3.4.3 Continuous Parameter and Multiparameter Markov Chains |
|
|
68 | (3) |
|
3.5 Markov Chain Monte Carlo Sampling |
|
|
71 | (10) |
|
3.5.1 Examples of Markov Chain Monte Carlo Calculations |
|
|
71 | (1) |
|
3.5.2 Metropolis-Hastings Algorithm |
|
|
72 | (5) |
|
|
77 | (4) |
|
4 Elementary Frequentist Statistics |
|
|
81 | (30) |
|
4.1 Introduction to Frequentist Statistics |
|
|
81 | (1) |
|
4.2 Means and Variances for Unweighted Data |
|
|
82 | (4) |
|
4.3 Data with Uncorrelated Measurement Errors |
|
|
86 | (5) |
|
4.4 Data with Correlated Measurement Errors |
|
|
91 | (4) |
|
4.5 Variance of the Variance and Students t Distribution |
|
|
95 | (6) |
|
4.5.1 Variance of the Variance |
|
|
96 | (2) |
|
4.5.2 Student's t Distribution |
|
|
98 | (2) |
|
|
100 | (1) |
|
4.6 Principal Component Analysis |
|
|
101 | (6) |
|
4.6.1 Correlation Coefficient |
|
|
101 | (1) |
|
4.6.2 Principal Component Analysis |
|
|
102 | (5) |
|
4.7 Kolmogorov-Smirnov Test |
|
|
107 | (4) |
|
4.7.1 One-Sample K-S Test |
|
|
107 | (2) |
|
4.7.2 Two-Sample K-S Test |
|
|
109 | (2) |
|
5 Linear Least Squares Estimation |
|
|
111 | (44) |
|
|
111 | (1) |
|
5.2 Likelihood Statistics |
|
|
112 | (8) |
|
5.2.1 Likelihood Function |
|
|
112 | (3) |
|
5.2.2 Maximum Likelihood Principle |
|
|
115 | (4) |
|
5.2.3 Relation to Least Squares and Χ2 Minimization |
|
|
119 | (1) |
|
5.3 Fits of Polynomials to Data |
|
|
120 | (17) |
|
|
120 | (6) |
|
5.3.2 Fits with Polynomials of Arbitrary Degree |
|
|
126 | (2) |
|
5.3.3 Variances, Covariances, and Biases |
|
|
128 | (8) |
|
5.3.4 Monte Carlo Error Analysis |
|
|
136 | (1) |
|
5.4 Need for Covariances and Propagation of Errors |
|
|
137 | (7) |
|
5.4.1 Need for Covariances |
|
|
137 | (2) |
|
5.4.2 Propagation of Errors |
|
|
139 | (3) |
|
5.4.3 Monte Carlo Error Propagation |
|
|
142 | (2) |
|
5.5 General Linear Least Squares |
|
|
144 | (8) |
|
5.5.1 Linear Least Squares with Nonpolynomial Functions |
|
|
144 | (3) |
|
5.5.2 Fits with Correlations among the Measurement Errors |
|
|
147 | (2) |
|
5.5.3 Χ2 Test for Goodness of Fit |
|
|
149 | (3) |
|
5.6 Fits with More Than One Dependent Variable |
|
|
152 | (3) |
|
6 Nonlinear Least Squares Estimation |
|
|
155 | (32) |
|
|
155 | (2) |
|
6.2 Linearization of Nonlinear Fits |
|
|
157 | (6) |
|
6.2.1 Data with Uncorrelated Measurement Errors |
|
|
158 | (3) |
|
6.2.2 Data with Correlated Measurement Errors |
|
|
161 | (1) |
|
6.2.3 Practical Considerations |
|
|
162 | (1) |
|
6.3 Other Methods for Minimizing S |
|
|
163 | (8) |
|
|
163 | (1) |
|
6.3.2 Method of Steepest Descent, Newton's Method, and Marquardt's Method |
|
|
164 | (4) |
|
6.3.3 Simplex Optimization |
|
|
168 | (1) |
|
6.3.4 Simulated Annealing |
|
|
168 | (3) |
|
|
171 | (5) |
|
6.4.1 Inversion of the Hessian Matrix |
|
|
171 | (2) |
|
6.4.2 Direct Calculation of the Covariance Matrix |
|
|
173 | (3) |
|
6.4.3 Summary and the Estimated Covariance Matrix |
|
|
176 | (1) |
|
|
176 | (5) |
|
6.6 Fits with Errors in Both the Dependent and Independent Variables |
|
|
181 | (6) |
|
6.6.1 Data with Uncorrelated Errors |
|
|
182 | (2) |
|
6.6.2 Data with Correlated Errors |
|
|
184 | (3) |
|
|
187 | (34) |
|
7.1 Introduction to Bayesian Statistics |
|
|
187 | (4) |
|
7.2 Single-Parameter Estimation: Means, Modes, and Variances |
|
|
191 | (8) |
|
|
191 | (1) |
|
7.2.2 Gaussian Priors and Likelihood Functions |
|
|
192 | (2) |
|
7.2.3 Binomial and Beta Distributions |
|
|
194 | (1) |
|
7.2.4 Poisson Distribution and Uniform Priors |
|
|
195 | (3) |
|
7.2.5 More about the Prior Probability Distribution |
|
|
198 | (1) |
|
7.3 Multiparameter Estimation |
|
|
199 | (15) |
|
7.3.1 Formal Description of the Problem |
|
|
199 | (1) |
|
7.3.2 Laplace Approximation |
|
|
200 | (3) |
|
7.3.3 Gaussian Likelihoods and Priors: Connection to Least Squares |
|
|
203 | (8) |
|
7.3.4 Difficult Posterior Distributions: Markov Chain Monte Carlo Sampling |
|
|
211 | (1) |
|
|
212 | (2) |
|
|
214 | (3) |
|
|
217 | (4) |
|
7.5.1 Prior Probability Distribution |
|
|
217 | (1) |
|
7.5.2 Likelihood Function |
|
|
218 | (1) |
|
7.5.3 Posterior Distribution Function |
|
|
218 | (1) |
|
7.5.4 Meaning of Probability |
|
|
219 | (1) |
|
|
219 | (2) |
|
8 Introduction to Fourier Analysis |
|
|
221 | (35) |
|
|
221 | (1) |
|
8.2 Complete Sets of Orthonormal Functions |
|
|
221 | (5) |
|
|
226 | (7) |
|
|
233 | (9) |
|
8.4.1 Fourier Transform Pairs |
|
|
234 | (7) |
|
8.4.2 Summary of Useful Fourier Transform Pairs |
|
|
241 | (1) |
|
8.5 Discrete Fourier Transform |
|
|
242 | (7) |
|
8.5.1 Derivation from the Continuous Fourier Transform |
|
|
243 | (2) |
|
8.5.2 Derivation from the Orthogonality Relations for Discretely Sampled Sine and Cosine Functions |
|
|
245 | (3) |
|
8.5.3 Parseval's Theorem and the Power Spectrum |
|
|
248 | (1) |
|
8.6 Convolution and the Convolution Theorem |
|
|
249 | (7) |
|
|
249 | (5) |
|
8.6.2 Convolution Theorem |
|
|
254 | (2) |
|
9 Analysis of Sequences: Power Spectra and Periodograms |
|
|
256 | (36) |
|
|
256 | (1) |
|
9.2 Continuous Sequences: Data Windows, Spectral Windows, and Aliasing |
|
|
256 | (9) |
|
9.2.1 Data Windows and Spectral Windows |
|
|
257 | (6) |
|
|
263 | (2) |
|
9.2.3 Arbitrary Data Windows |
|
|
265 | (1) |
|
|
265 | (6) |
|
9.3.1 The Need to Oversample Fm |
|
|
266 | (1) |
|
|
267 | (3) |
|
9.3.3 Integration Sampling |
|
|
270 | (1) |
|
|
271 | (7) |
|
9.4.1 Deterministic and Stochastic Processes |
|
|
271 | (1) |
|
9.4.2 Power Spectrum of White Noise |
|
|
272 | (3) |
|
9.4.3 Deterministic Signals in the Presence of Noise |
|
|
275 | (2) |
|
9.4.4 Nonwhite, Non-Gaussian Noise |
|
|
277 | (1) |
|
9.5 Sequences with Uneven Spacing |
|
|
278 | (9) |
|
9.5.1 Least Squares Periodogram |
|
|
278 | (2) |
|
9.5.2 Lomb-Scargle Periodogram |
|
|
280 | (4) |
|
9.5.3 Generalized Lomb-Scargle Periodogram |
|
|
284 | (3) |
|
9.6 Signals with Variable Periods: The O-C Diagram |
|
|
287 | (5) |
|
10 Analysis of Sequences: Convolution and Covariance |
|
|
292 | (41) |
|
10.1 Convolution Revisited |
|
|
292 | (9) |
|
10.1.1 Impulse Response Function |
|
|
292 | (5) |
|
10.1.2 Frequency Response Function |
|
|
297 | (4) |
|
10.2 Deconvolution and Data Reconstruction |
|
|
301 | (8) |
|
10.2.1 Effect of Noise on Deconvolution |
|
|
301 | (4) |
|
10.2.2 Wiener Deconvolution |
|
|
305 | (3) |
|
10.2.3 Richardson-Lucy Algorithm |
|
|
308 | (1) |
|
10.3 Autocovariance Functions |
|
|
309 | (15) |
|
10.3.1 Basic Properties of Autocovariance Functions |
|
|
309 | (4) |
|
10.3.2 Relation to the Power Spectrum |
|
|
313 | (3) |
|
10.3.3 Application to Stochastic Processes |
|
|
316 | (8) |
|
10.4 Cross-Covariance Functions |
|
|
324 | (9) |
|
10.4.1 Basic Properties of Cross-Covariance Functions |
|
|
324 | (2) |
|
10.4.2 Relation to Χ2 and to the Cross Spectrum |
|
|
326 | (3) |
|
10.4.3 Detection of Pulsed Signals in Noise |
|
|
329 | (4) |
|
|
|
A Some Useful Definite Integrals |
|
|
333 | (5) |
|
B Method of Lagrange Multipliers |
|
|
338 | (4) |
|
C Additional Properties of the Gaussian Probability Distribution |
|
|
342 | (10) |
|
D The n-Dimensional Sphere |
|
|
352 | (2) |
|
E Review of Linear Algebra and Matrices |
|
|
354 | (20) |
|
F Limit of [ 1 + f(x)/n]n for Large n |
|
|
374 | (1) |
|
G Greens Function Solutions for Impulse Response Functions |
|
|
375 | (4) |
|
H Second-Order Autoregressive Process |
|
|
379 | (4) |
Bibliography |
|
383 | (2) |
Index |
|
385 | |