Foreword |
|
xv | |
Preface |
|
xvii | |
About the Authors |
|
xxi | |
|
|
1 | (124) |
|
1 The Big (Bayesian) Picture |
|
|
3 | (14) |
|
1.1 Thinking like a Bayesian |
|
|
4 | (5) |
|
|
5 | (1) |
|
1.1.2 The meaning of probability |
|
|
6 | (1) |
|
1.1.3 The Bayesian balancing act |
|
|
6 | (2) |
|
|
8 | (1) |
|
1.2 A quick history lesson |
|
|
9 | (2) |
|
|
11 | (3) |
|
1.3.1 Unit 1: Bayesian foundations |
|
|
11 | (1) |
|
1.3.2 Unit 2: Posterior simulation & analysis |
|
|
12 | (1) |
|
1.3.3 Unit 3: Bayesian regression & classification |
|
|
12 | (1) |
|
1.3.4 Unit 4: Hierarchical Bayesian models |
|
|
13 | (1) |
|
|
14 | (1) |
|
|
14 | (3) |
|
|
17 | (32) |
|
2.1 Building a Bayesian model for events |
|
|
19 | (11) |
|
2.1.1 Prior probability model |
|
|
19 | (1) |
|
2.1.2 Conditional probability & likelihood |
|
|
20 | (2) |
|
2.1.3 Normalizing constants |
|
|
22 | (2) |
|
2.1.4 Posterior probability model via Bayes' Rule! |
|
|
24 | (1) |
|
2.1.5 Posterior simulation |
|
|
25 | (5) |
|
2.2 Example: Pop vs soda vs coke |
|
|
30 | (1) |
|
2.3 Building a Bayesian model for random variables |
|
|
31 | (11) |
|
2.3.1 Prior probability model |
|
|
32 | (1) |
|
2.3.2 The Binomial data model |
|
|
33 | (2) |
|
2.3.3 The Binomial likelihood function |
|
|
35 | (1) |
|
2.3.4 Normalizing constant |
|
|
36 | (1) |
|
2.3.5 Posterior probability model |
|
|
37 | (1) |
|
|
38 | (2) |
|
2.3.7 Posterior simulation |
|
|
40 | (2) |
|
|
42 | (1) |
|
|
42 | (7) |
|
2.5.1 Building up to Bayes' Rule |
|
|
42 | (1) |
|
2.5.2 Practice Bayes' Rule for events |
|
|
43 | (2) |
|
2.5.3 Practice Bayes' Rule for random variables |
|
|
45 | (2) |
|
2.5.4 Simulation exercises |
|
|
47 | (2) |
|
3 The Beta-Binomial Bayesian Model |
|
|
49 | (26) |
|
|
50 | (5) |
|
|
51 | (3) |
|
3.1.2 Tuning the Beta prior |
|
|
54 | (1) |
|
3.2 The Binomial data model & likelihood function |
|
|
55 | (2) |
|
3.3 The Beta posterior model |
|
|
57 | (4) |
|
3.4 The Beta-Binomial model |
|
|
61 | (2) |
|
3.5 Simulating the Beta-Binomial |
|
|
63 | (1) |
|
3.6 Example: Milgram's behavioral study of obedience |
|
|
64 | (3) |
|
3.6.1 A Bayesian analysis |
|
|
65 | (1) |
|
3.6.2 The role of ethics in statistics and data science |
|
|
66 | (1) |
|
|
67 | (1) |
|
|
68 | (7) |
|
3.8.1 Practice: Beta prior models |
|
|
68 | (3) |
|
3.8.2 Practice: Beta-Binomial models |
|
|
71 | (4) |
|
4 Balance and Sequentiality in Bayesian Analyses |
|
|
75 | (22) |
|
4.1 Different priors, different posteriors |
|
|
77 | (3) |
|
4.2 Different data, different posteriors |
|
|
80 | (2) |
|
4.3 Striking a balance between the prior & data |
|
|
82 | (3) |
|
4.3.1 Connecting observations to concepts |
|
|
82 | (1) |
|
4.3.2 Connecting concepts to theory |
|
|
83 | (2) |
|
4.4 Sequential analysis: Evolving with data |
|
|
85 | (3) |
|
4.5 Proving data order invariance |
|
|
88 | (1) |
|
|
89 | (1) |
|
4.7 A note on subjectivity |
|
|
90 | (1) |
|
|
91 | (1) |
|
|
92 | (5) |
|
|
92 | (1) |
|
4.9.2 Practice: Different priors, different posteriors |
|
|
93 | (1) |
|
4.9.3 Practice: Balancing the data k prior |
|
|
93 | (2) |
|
4.9.4 Practice: Sequentiality |
|
|
95 | (2) |
|
|
97 | (28) |
|
5.1 Revisiting choice of prior |
|
|
97 | (3) |
|
5.2 Gamma-Poisson conjugate family |
|
|
100 | (9) |
|
5.2.1 The Poisson data model |
|
|
100 | (3) |
|
|
103 | (1) |
|
|
104 | (2) |
|
5.2.4 Gamma-Poisson conjugacy |
|
|
106 | (3) |
|
5.3 Normal-Normal conjugate family |
|
|
109 | (8) |
|
5.3.1 The Normal data model |
|
|
109 | (2) |
|
|
111 | (2) |
|
5.3.3 Normal-Normal conjugacy |
|
|
113 | (3) |
|
5.3.4 Optional: Proving Normal-Normal conjugacy |
|
|
116 | (1) |
|
5.4 Why no simulation in this chapter? |
|
|
117 | (1) |
|
5.5 Critiques of conjugate family models |
|
|
118 | (1) |
|
|
118 | (1) |
|
|
118 | (7) |
|
5.7.1 Practice: Gamma-Poisson |
|
|
118 | (2) |
|
5.7.2 Practice: Normal-Normal |
|
|
120 | (2) |
|
5.7.3 General practice exercises |
|
|
122 | (3) |
|
II Posterior Simulation & Analysis |
|
|
125 | (84) |
|
6 Approximating the Posterior |
|
|
127 | (32) |
|
|
129 | (8) |
|
6.1.1 A Beta-Binomial example |
|
|
129 | (5) |
|
6.1.2 A Gamma-Poisson example |
|
|
134 | (2) |
|
|
136 | (1) |
|
6.2 Markov chains via rstan |
|
|
137 | (8) |
|
6.2.1 A Beta-Binomial example |
|
|
139 | (4) |
|
6.2.2 A Gamma-Poisson example |
|
|
143 | (2) |
|
6.3 Markov chain diagnostics |
|
|
145 | (10) |
|
6.3.1 Examining trace plots |
|
|
146 | (1) |
|
6.3.2 Comparing parallel chains |
|
|
147 | (1) |
|
6.3.3 Calculating effective sample size & autocorrelation |
|
|
148 | (5) |
|
|
153 | (2) |
|
|
155 | (1) |
|
|
156 | (3) |
|
6.5.1 Conceptual exercises |
|
|
156 | (1) |
|
6.5.2 Practice: Grid approximation |
|
|
156 | (1) |
|
|
157 | (2) |
|
|
159 | (24) |
|
|
159 | (5) |
|
7.2 The Metropolis-Hastings algorithm |
|
|
164 | (4) |
|
7.3 Implementing the Metropolis-Hastings |
|
|
168 | (2) |
|
7.4 Tuning the Metropolis-Hastings algorithm |
|
|
170 | (2) |
|
7.5 A Beta-Binomial example |
|
|
172 | (3) |
|
7.6 Why the algorithm works |
|
|
175 | (1) |
|
7.7 Variations on the theme |
|
|
176 | (1) |
|
|
176 | (1) |
|
|
176 | (7) |
|
7.9.1 Conceptual exercises |
|
|
177 | (1) |
|
7.9.2 Practice: Normal-Normal simulation |
|
|
178 | (2) |
|
7.9.3 Practice: Simulating more Bayesian models |
|
|
180 | (3) |
|
8 Posterior Inference & Prediction |
|
|
183 | (26) |
|
|
184 | (3) |
|
8.2 Posterior hypothesis testing |
|
|
187 | (5) |
|
|
187 | (4) |
|
|
191 | (1) |
|
|
192 | (3) |
|
8.4 Posterior analysis with MCMC |
|
|
195 | (5) |
|
8.4.1 Posterior simulation |
|
|
195 | (1) |
|
8.4.2 Posterior estimation k hypothesis testing |
|
|
196 | (3) |
|
8.4.3 Posterior prediction |
|
|
199 | (1) |
|
|
200 | (1) |
|
|
201 | (1) |
|
|
202 | (7) |
|
8.7.1 Conceptual exercises |
|
|
202 | (1) |
|
|
202 | (2) |
|
|
204 | (5) |
|
III Bayesian Regression & Classification |
|
|
209 | (164) |
|
9 Simple Normal Regression |
|
|
211 | (32) |
|
9.1 Building the regression model |
|
|
213 | (3) |
|
9.1.1 Specifying the data model |
|
|
213 | (2) |
|
9.1.2 Specifying the priors |
|
|
215 | (1) |
|
9.1.3 Putting it all together |
|
|
216 | (1) |
|
9.2 Tuning prior models for regression parameters |
|
|
216 | (3) |
|
|
219 | (4) |
|
9.3.1 Simulation via rstanarm |
|
|
220 | (2) |
|
9.3.2 Optional: Simulation via rstan |
|
|
222 | (1) |
|
9.4 Interpreting the posterior |
|
|
223 | (3) |
|
|
226 | (5) |
|
9.5.1 Building a posterior predictive model |
|
|
227 | (2) |
|
9.5.2 Posterior prediction with rstanarm |
|
|
229 | (2) |
|
9.6 Sequential regression modeling |
|
|
231 | (1) |
|
9.7 Using default rstanarm priors |
|
|
232 | (3) |
|
|
235 | (1) |
|
|
236 | (1) |
|
|
236 | (7) |
|
9.10.1 Conceptual exercises |
|
|
236 | (1) |
|
|
237 | (6) |
|
10 Evaluating Regression Models |
|
|
243 | (24) |
|
|
243 | (2) |
|
10.2 How wrong is the model? |
|
|
245 | (5) |
|
10.2.1 Checking the model assumptions |
|
|
245 | (3) |
|
10.2.2 Dealing with wrong models |
|
|
248 | (2) |
|
10.3 How accurate are the posterior predictive models? |
|
|
250 | (10) |
|
10.3.1 Posterior predictive summaries |
|
|
251 | (4) |
|
|
255 | (4) |
|
10.3.3 Expected log-predictive density |
|
|
259 | (1) |
|
10.3.4 Improving posterior predictive accuracy |
|
|
260 | (1) |
|
10.4 How good is the MCMC simulation vs how good is the model? |
|
|
260 | (1) |
|
|
261 | (1) |
|
|
261 | (6) |
|
10.6.1 Conceptual exercises |
|
|
261 | (2) |
|
|
263 | (2) |
|
10.6.3 Open-ended exercises |
|
|
265 | (2) |
|
11 Extending the Normal Regression Model |
|
|
267 | (36) |
|
11.1 Utilizing a categorical predictor |
|
|
270 | (4) |
|
11.1.1 Building the model |
|
|
271 | (1) |
|
11.1.2 Simulating the posterior |
|
|
272 | (2) |
|
11.2 Utilizing two predictors |
|
|
274 | (6) |
|
11.2.1 Building the model |
|
|
275 | (1) |
|
11.2.2 Understanding the priors |
|
|
276 | (1) |
|
11.2.3 Simulating the posterior |
|
|
277 | (2) |
|
11.2.4 Posterior prediction |
|
|
279 | (1) |
|
11.3 Optional: Utilizing interaction terms |
|
|
280 | (6) |
|
11.3.1 Building the model |
|
|
280 | (1) |
|
11.3.2 Simulating the posterior |
|
|
281 | (2) |
|
11.3.3 Do you need an interaction term? |
|
|
283 | (3) |
|
11.4 Dreaming bigger: Utilizing more than 2 predictors! |
|
|
286 | (3) |
|
11.5 Model evaluation & comparison |
|
|
289 | (9) |
|
11.5.1 Evaluating predictive accuracy using visualizations |
|
|
290 | (2) |
|
11.5.2 Evaluating predictive accuracy using cross-validation |
|
|
292 | (1) |
|
11.5.3 Evaluating predictive accuracy using ELPD |
|
|
293 | (1) |
|
11.5.4 The bias-variance trade-off |
|
|
294 | (4) |
|
|
298 | (1) |
|
|
299 | (4) |
|
11.7.1 Conceptual exercises |
|
|
299 | (1) |
|
|
300 | (2) |
|
11.7.3 Open-ended exercises |
|
|
302 | (1) |
|
12 Poisson & Negative Binomial Regression |
|
|
303 | (26) |
|
12.1 Building the Poisson regression model |
|
|
306 | (6) |
|
12.1.1 Specifying the data model |
|
|
306 | (4) |
|
12.1.2 Specifying the priors |
|
|
310 | (2) |
|
12.2 Simulating the posterior |
|
|
312 | (1) |
|
12.3 Interpreting the posterior |
|
|
313 | (2) |
|
12.4 Posterior prediction |
|
|
315 | (2) |
|
|
317 | (2) |
|
12.6 Negative Binomial regression for overdispersed counts |
|
|
319 | (5) |
|
12.7 Generalized linear models: Building on the theme |
|
|
324 | (1) |
|
|
325 | (1) |
|
|
326 | (3) |
|
12.9.1 Conceptual exercises |
|
|
326 | (1) |
|
|
327 | (2) |
|
|
329 | (26) |
|
13.1 Pause: Odds & probability |
|
|
330 | (1) |
|
13.2 Building the logistic regression model |
|
|
331 | (5) |
|
13.2.1 Specifying the data model |
|
|
331 | (3) |
|
13.2.2 Specifying the priors |
|
|
334 | (2) |
|
13.3 Simulating the posterior |
|
|
336 | (3) |
|
13.4 Prediction & classification |
|
|
339 | (2) |
|
|
341 | (5) |
|
|
346 | (2) |
|
|
348 | (1) |
|
|
349 | (6) |
|
13.8.1 Conceptual exercises |
|
|
349 | (1) |
|
|
350 | (2) |
|
13.8.3 Open-ended exercises |
|
|
352 | (3) |
|
14 Naive Bayes Classification |
|
|
355 | (18) |
|
14.1 Classifying one penguin |
|
|
356 | (9) |
|
14.1.1 One categorical predictor |
|
|
357 | (2) |
|
14.1.2 One quantitative predictor |
|
|
359 | (3) |
|
|
362 | (3) |
|
14.2 Implementing & evaluating naive Bayes classification |
|
|
365 | (4) |
|
14.3 Naive Bayes vs logistic regression |
|
|
369 | (1) |
|
|
369 | (1) |
|
|
370 | (3) |
|
14.5.1 Conceptual exercises |
|
|
370 | (1) |
|
|
370 | (2) |
|
14.5.3 Open-ended exercises |
|
|
372 | (1) |
|
IV Hierarchical Bayesian models |
|
|
373 | (138) |
|
15 Hierarchical Models are Exciting |
|
|
375 | (12) |
|
|
377 | (3) |
|
|
380 | (2) |
|
|
382 | (1) |
|
15.4 Partial pooling with hierarchical models |
|
|
383 | (1) |
|
|
384 | (1) |
|
|
385 | (2) |
|
15.6.1 Conceptual exercises |
|
|
385 | (1) |
|
|
385 | (2) |
|
16 (Normal) Hierarchical Models without Predictors |
|
|
387 | (34) |
|
16.1 Complete pooled model |
|
|
390 | (3) |
|
|
393 | (4) |
|
16.3 Building the hierarchical model |
|
|
397 | (4) |
|
|
397 | (2) |
|
16.3.2 Another way to think about it |
|
|
399 | (1) |
|
16.3.3 Within- vs between-group variability |
|
|
400 | (1) |
|
|
401 | (6) |
|
16.4.1 Posterior simulation |
|
|
401 | (2) |
|
16.4.2 Posterior analysis of global parameters |
|
|
403 | (1) |
|
16.4.3 Posterior analysis of group-specific parameters |
|
|
404 | (3) |
|
16.5 Posterior prediction |
|
|
407 | (4) |
|
16.6 Shrinkage & the bias-variance trade-off |
|
|
411 | (3) |
|
16.7 Not everything is hierarchical |
|
|
414 | (2) |
|
|
416 | (1) |
|
|
417 | (4) |
|
16.9.1 Conceptual exercises |
|
|
417 | (1) |
|
|
418 | (3) |
|
17 (Normal) Hierarchical Models with Predictors |
|
|
421 | (42) |
|
17.1 First steps: Complete pooling |
|
|
422 | (1) |
|
17.2 Hierarchical model with varying intercepts |
|
|
423 | (12) |
|
|
423 | (4) |
|
17.2.2 Another way to think about it |
|
|
427 | (1) |
|
|
427 | (2) |
|
17.2.4 Posterior simulation & analysis |
|
|
429 | (6) |
|
17.3 Hierarchical model with varying intercepts & slopes |
|
|
435 | (12) |
|
|
436 | (4) |
|
17.3.2 Optional: The decomposition of covariance model |
|
|
440 | (2) |
|
17.3.3 Posterior simulation & analysis |
|
|
442 | (5) |
|
17.4 Model evaluation & selection |
|
|
447 | (3) |
|
17.5 Posterior prediction |
|
|
450 | (2) |
|
17.6 Details: Longitudinal data |
|
|
452 | (1) |
|
17.7 Example: Danceability |
|
|
452 | (5) |
|
|
457 | (1) |
|
|
458 | (5) |
|
17.9.1 Conceptual exercises |
|
|
458 | (1) |
|
|
459 | (2) |
|
17.9.3 Open-ended exercises |
|
|
461 | (2) |
|
18 Non-Normal Hierarchical Regression & Classification |
|
|
463 | (22) |
|
18.1 Hierarchical logistic regression |
|
|
463 | (10) |
|
18.1.1 Model building & simulation |
|
|
466 | (3) |
|
18.1.2 Posterior analysis |
|
|
469 | (1) |
|
18.1.3 Posterior classification |
|
|
470 | (2) |
|
|
472 | (1) |
|
18.2 Hierarchical Poisson & Negative Binomial regression |
|
|
473 | (7) |
|
18.2.1 Model building & simulation |
|
|
474 | (3) |
|
18.2.2 Posterior analysis |
|
|
477 | (2) |
|
|
479 | (1) |
|
|
480 | (1) |
|
|
480 | (5) |
|
18.4.1 Applied & conceptual exercises |
|
|
480 | (3) |
|
18.4.2 Open-ended exercises |
|
|
483 | (2) |
|
|
485 | (26) |
|
19.1 Group-level predictors |
|
|
485 | (12) |
|
19.1.1 A model using only individual-level predictors |
|
|
486 | (3) |
|
19.1.2 Incorporating group-level predictors |
|
|
489 | (3) |
|
19.1.3 Posterior simulation & global analysis |
|
|
492 | (2) |
|
19.1.4 Posterior group-level analysis |
|
|
494 | (3) |
|
19.1.5 We're just scratching the surface! |
|
|
497 | (1) |
|
19.2 Incorporating two (or more!) grouping variables |
|
|
497 | (8) |
|
19.2.1 Data with two grouping variables |
|
|
497 | (2) |
|
19.2.2 Building a model with two grouping variables |
|
|
499 | (2) |
|
19.2.3 Simulating models with two grouping variables |
|
|
501 | (2) |
|
19.2.4 Examining the group-specific parameters |
|
|
503 | (2) |
|
19.2.5 We're just scratching the surface! |
|
|
505 | (1) |
|
|
505 | (4) |
|
19.3.1 Conceptual exercises |
|
|
505 | (1) |
|
|
506 | (3) |
|
|
509 | (2) |
Bibliography |
|
511 | (6) |
Index |
|
517 | |