Part I Building Blocks for Bayesian Networks |
|
|
|
3 | (16) |
|
1.1 An Example Bayes Network |
|
|
4 | (3) |
|
1.2 Cognitively Diagnostic Assessment |
|
|
7 | (4) |
|
1.3 Cognitive and Psychometric Science |
|
|
11 | (3) |
|
1.4 Ten Reasons for Considering Bayesian Networks |
|
|
14 | (2) |
|
|
16 | (3) |
|
2 An Introduction to Evidence-Centered Design |
|
|
19 | (22) |
|
|
20 | (1) |
|
2.2 Assessment as Evidentiary Argument |
|
|
21 | (2) |
|
2.3 The Process of Design |
|
|
23 | (3) |
|
|
26 | (13) |
|
2.4.1 The Conceptual Assessment Framework |
|
|
27 | (7) |
|
2.4.2 Four-Process Architecture for Assessment Delivery |
|
|
34 | (4) |
|
2.4.3 Pretesting and Calibration |
|
|
38 | (1) |
|
|
39 | (2) |
|
3 Bayesian Probability and Statistics: a Review |
|
|
41 | (40) |
|
3.1 Probability: Objective and Subjective |
|
|
41 | (5) |
|
3.1.1 Objective Notions of Probability |
|
|
42 | (1) |
|
3.1.2 Subjective Notions of Probability |
|
|
43 | (2) |
|
3.1.3 Subjective-Objective Probability |
|
|
45 | (1) |
|
3.2 Conditional Probability |
|
|
46 | (5) |
|
3.3 Independence and Conditional Independence |
|
|
51 | (6) |
|
3.3.1 Conditional Independence |
|
|
53 | (1) |
|
3.3.2 Common Variable Dependence |
|
|
54 | (1) |
|
3.3.3 Competing Explanations |
|
|
55 | (2) |
|
|
57 | (5) |
|
3.4.1 The Probability Mass and Density Functions |
|
|
57 | (3) |
|
3.4.2 Expectation and Variance |
|
|
60 | (2) |
|
|
62 | (19) |
|
3.5.1 Re-expressing Bayes Theorem |
|
|
63 | (1) |
|
|
63 | (4) |
|
|
67 | (5) |
|
|
72 | (2) |
|
3.5.5 Noninformative Priors |
|
|
74 | (2) |
|
3.5.6 Evidence-Centered Design and the Bayesian Paradigm |
|
|
76 | (5) |
|
4 Basic Graph Theory and Graphical Models |
|
|
81 | (24) |
|
|
82 | (4) |
|
4.1.1 Simple Undirected Graphs |
|
|
83 | (1) |
|
|
83 | (1) |
|
|
84 | (2) |
|
4.2 Factorization of the Joint Distribution |
|
|
86 | (5) |
|
4.2.1 Directed Graph Representation |
|
|
86 | (2) |
|
4.2.2 Factorization Hypergraphs |
|
|
88 | (2) |
|
4.2.3 Undirected Graphical Representation |
|
|
90 | (1) |
|
4.3 Separation and Conditional Independence |
|
|
91 | (4) |
|
4.3.1 Separation and D-Separation |
|
|
91 | (2) |
|
4.3.2 Reading Dependence and Independence from Graphs |
|
|
93 | (1) |
|
4.3.3 Gibbs-Markov Equivalence Theorem |
|
|
94 | (1) |
|
4.4 Edge Directions and Causality |
|
|
95 | (2) |
|
4.5 Other Representations |
|
|
97 | (8) |
|
|
97 | (2) |
|
4.5.2 Structural Equation Models |
|
|
99 | (1) |
|
4.5.3 Other Graphical Models |
|
|
100 | (5) |
|
|
105 | (52) |
|
5.1 Belief Updating with Two Variables |
|
|
106 | (5) |
|
5.2 More Efficient Procedures for Chains and Trees |
|
|
111 | (11) |
|
5.2.1 Propagation in Chains |
|
|
112 | (4) |
|
5.2.2 Propagation in Trees |
|
|
116 | (3) |
|
|
119 | (3) |
|
5.3 Belief Updating in Multiply Connected Graphs |
|
|
122 | (13) |
|
5.3.1 Updating in the Presence of Loops |
|
|
122 | (1) |
|
5.3.2 Constructing a Junction Tree |
|
|
123 | (11) |
|
5.3.3 Propagating Evidence Through a Junction Tree |
|
|
134 | (1) |
|
5.4 Application to Assessment |
|
|
135 | (10) |
|
5.4.1 Proficiency and Evidence Model Bayes Net_Fragments |
|
|
137 | (2) |
|
5.4.2 Junction Trees for Fragments |
|
|
139 | (4) |
|
5.4.3 Calculation with Fragments |
|
|
143 | (2) |
|
5.5 The Structure of a Test |
|
|
145 | (4) |
|
5.5.1 The Q-Matrix for Assessments Using Only Discrete Items |
|
|
146 | (1) |
|
5.5.2 The Q-Matrix for a Test Using Multi-observable Tasks |
|
|
147 | (2) |
|
5.6 Alternative Computing Algorithms |
|
|
149 | (8) |
|
5.6.1 Variants of the Propagation Algorithm |
|
|
150 | (1) |
|
5.6.2 Dealing with Unfavorable Topologies |
|
|
150 | (7) |
|
|
157 | (40) |
|
|
158 | (8) |
|
6.1.1 General Features of the IRT Bayes Net |
|
|
161 | (1) |
|
6.1.2 Inferences in the IRT Bayes Net |
|
|
162 | (4) |
|
|
166 | (6) |
|
6.3 Compensatory, Conjunctive, and Disjunctive Models |
|
|
172 | (6) |
|
6.4 A Binary-Skills Measurement Model |
|
|
178 | (12) |
|
6.4.1 The Domain of Mixed Number Subtraction |
|
|
178 | (2) |
|
6.4.2 A Bayes Net Model for Mixed-Number Subtraction |
|
|
180 | (4) |
|
6.4.3 Inferences from the Mixed-Number Subtraction Bayes Net |
|
|
184 | (6) |
|
|
190 | (7) |
|
7 Explanation and Test Construction |
|
|
197 | (44) |
|
7.1 Simple Explanation Techniques |
|
|
198 | (3) |
|
|
198 | (2) |
|
7.1.2 Most Likely Scenario |
|
|
200 | (1) |
|
|
201 | (8) |
|
7.2.1 Evidence Balance Sheet |
|
|
202 | (3) |
|
7.2.2 Evidence Flow Through the Graph |
|
|
205 | (4) |
|
|
209 | (6) |
|
7.3.1 Value of Information |
|
|
209 | (4) |
|
7.3.2 Expected Weight of Evidence |
|
|
213 | (2) |
|
|
215 | (1) |
|
|
215 | (9) |
|
7.4.1 Computer Adaptive Testing |
|
|
216 | (1) |
|
|
217 | (3) |
|
|
220 | (4) |
|
7.5 Reliability and Assessment Information |
|
|
224 | (17) |
|
|
225 | (5) |
|
|
230 | (1) |
|
7.5.3 Expected Value Matrix |
|
|
230 | (2) |
|
7.5.4 Weight of Evidence as Information |
|
|
232 | (9) |
Part II Learning and Revising Models from Data |
|
|
8 Parameters for Bayesian Network Models |
|
|
241 | (38) |
|
8.1 Parameterizing a Graphical Model |
|
|
241 | (3) |
|
|
244 | (2) |
|
8.3 The Conditional Multinomial-Hyper-Dirichlet Family |
|
|
246 | (4) |
|
8.3.1 Beta-Binomial Family |
|
|
247 | (1) |
|
8.3.2 Dirichlet-Multinomial Family |
|
|
248 | (1) |
|
8.3.3 The Hyper-Dirichlet Law |
|
|
248 | (2) |
|
8.4 Noisy-OR and Noisy-AND Models |
|
|
250 | (4) |
|
8.4.1 Separable Influence |
|
|
254 | (1) |
|
8.5 DiBello's Effective Theta Distributions |
|
|
254 | (13) |
|
8.5.1 Mapping Parent Skills to θ Space |
|
|
256 | (1) |
|
8.5.2 Combining Input Skills |
|
|
257 | (3) |
|
8.5.3 Samejima's Graded Response Model |
|
|
260 | (3) |
|
8.5.4 Normal Link Function |
|
|
263 | (4) |
|
8.6 Eliciting Parameters and Laws |
|
|
267 | (12) |
|
8.6.1 Eliciting Conditional Multinomial and Noisy-AND |
|
|
269 | (3) |
|
8.6.2 Priors for DiBello's Effective Theta Distributions |
|
|
272 | (1) |
|
|
273 | (6) |
|
9 Learning in Models with Fixed Structure |
|
|
279 | (52) |
|
9.1 Data, Models, and Plate Notation |
|
|
279 | (8) |
|
|
280 | (2) |
|
9.1.2 A Bayesian Framework for a Generic Measurement Model |
|
|
282 | (2) |
|
9.1.3 Extension to Covariates |
|
|
284 | (3) |
|
9.2 Techniques for Learning with Fixed Structure |
|
|
287 | (10) |
|
9.2.1 Bayesian Inference for the General Measurement Model |
|
|
288 | (1) |
|
9.2.2 Complete Data Tables |
|
|
289 | (8) |
|
9.3 Latent Variables as Missing Data |
|
|
297 | (1) |
|
|
298 | (7) |
|
9.5 Markov Chain Monte Carlo Estimation |
|
|
305 | (10) |
|
|
308 | (1) |
|
9.5.2 Properties of MCMC Estimation |
|
|
309 | (3) |
|
9.5.3 The Metropolis-Hastings Algorithm |
|
|
312 | (3) |
|
9.6 MCMC Estimation in Bayes Nets in Assessment |
|
|
315 | (9) |
|
9.6.1 Initial Calibration |
|
|
316 | (5) |
|
|
321 | (3) |
|
9.7 Caution: MCMC and EM are Dangerous! |
|
|
324 | (7) |
|
10 Critiquing and Learning Model Structure |
|
|
331 | (40) |
|
10.1 Fit Indices Based on Prediction Accuracy |
|
|
332 | (3) |
|
10.2 Posterior Predictive Checks |
|
|
335 | (7) |
|
|
342 | (5) |
|
10.4 Differential Task Functioning |
|
|
347 | (3) |
|
|
350 | (4) |
|
|
350 | (3) |
|
10.5.2 Prediction Criteria |
|
|
353 | (1) |
|
|
354 | (4) |
|
10.6.1 Simple Search Strategies |
|
|
355 | (1) |
|
|
356 | (1) |
|
|
357 | (1) |
|
10.6.4 Priors Over Models |
|
|
357 | (1) |
|
10.7 Equivalent Models and Causality |
|
|
358 | (4) |
|
|
358 | (1) |
|
10.7.2 Unobserved Variables |
|
|
358 | (2) |
|
10.7.3 Why Unsupervised Learning cannot Prove Causality |
|
|
360 | (2) |
|
|
362 | (9) |
|
11 An Illustrative Example |
|
|
371 | (40) |
|
11.1 Representing the Cognitive Model |
|
|
372 | (10) |
|
11.1.1 Representing the Cognitive Model as a Bayesian Network |
|
|
372 | (5) |
|
11.1.2 Representing the Cognitive Model as a Bayesian Network |
|
|
377 | (2) |
|
11.1.3 Higher-Level Structure of the Proficiency Model; i.e., p(θ/λ) and p(λ) |
|
|
379 | (2) |
|
11.1.4 High Level Structure of the Evidence Models; i.e., p(r) |
|
|
381 | (1) |
|
11.1.5 Putting the Pieces Together |
|
|
382 | (1) |
|
11.2 Calibrating the Model with Field Data |
|
|
382 | (15) |
|
|
383 | (6) |
|
|
389 | (3) |
|
11.2.3 Online Calibration |
|
|
392 | (5) |
|
|
397 | (8) |
|
11.3.1 Observable Characteristic Plots |
|
|
398 | (3) |
|
11.3.2 Posterior Predictive Checks |
|
|
401 | (4) |
|
|
405 | (6) |
Part III Evidence-Centered Assessment Design |
|
|
12 The Conceptual Assessment Framework |
|
|
411 | (56) |
|
12.1 Phases of the Design Process and Evidentiary Arguments |
|
|
414 | (10) |
|
12.1.1 Domain Analysis and Domain Modeling |
|
|
414 | (4) |
|
12.1.2 Arguments and Claims |
|
|
418 | (6) |
|
12.2 The Student Proficiency Model |
|
|
424 | (14) |
|
12.2.1 Proficiency Variables |
|
|
424 | (4) |
|
12.2.2 Relationships Among Proficiency Variables |
|
|
428 | (5) |
|
|
433 | (5) |
|
|
438 | (5) |
|
|
443 | (10) |
|
12.4.1 Rules of Evidence (for Evidence Identification) |
|
|
444 | (4) |
|
12.4.2 Statistical Models of Evidence (for Evidence Accumulation) |
|
|
448 | (5) |
|
|
453 | (5) |
|
12.6 The Presentation Model |
|
|
458 | (2) |
|
|
460 | (1) |
|
12.8 Putting It All Together |
|
|
461 | (6) |
|
13 The Evidence Accumulation Process |
|
|
467 | (40) |
|
13.1 The Four-Process Architecture |
|
|
468 | (6) |
|
13.1.1 A Simple Example of the Four-Process Framework |
|
|
471 | (3) |
|
13.2 Producing an Assessment |
|
|
474 | (14) |
|
13.2.1 Tasks and Task Model Variables |
|
|
474 | (4) |
|
|
478 | (8) |
|
13.2.3 Evidence Models, Links, and Calibration |
|
|
486 | (2) |
|
|
488 | (19) |
|
13.3.1 Basic Scoring Protocols |
|
|
489 | (4) |
|
|
493 | (4) |
|
13.3.3 Technical Considerations |
|
|
497 | (3) |
|
|
500 | (7) |
|
14 Biomass: An Assessment of Science Standards |
|
|
507 | (42) |
|
|
507 | (3) |
|
|
510 | (5) |
|
14.2.1 Reconceiving Standards |
|
|
510 | (3) |
|
|
513 | (1) |
|
|
514 | (1) |
|
14.3 The Biomass Conceptual Assessment Framework |
|
|
515 | (20) |
|
14.3.1 The Proficiency Model |
|
|
515 | (4) |
|
14.3.2 The Assembly Model |
|
|
519 | (4) |
|
|
523 | (6) |
|
|
529 | (6) |
|
14.4 The Assessment Delivery Processes |
|
|
535 | (10) |
|
14.4.1 Biomass Architecture |
|
|
536 | (2) |
|
14.4.2 The Presentation Process |
|
|
538 | (2) |
|
14.4.3 Evidence Identification |
|
|
540 | (1) |
|
14.4.4 Evidence Accumulation |
|
|
541 | (2) |
|
14.4.5 Activity Selection |
|
|
543 | (1) |
|
14.4.6 The Task/Evidence Composite Library |
|
|
543 | (1) |
|
14.4.7 Controlling the Flow of Information Among the Processes |
|
|
544 | (1) |
|
|
545 | (4) |
|
15 The Biomass Measurement Model |
|
|
549 | (34) |
|
15.1 Specifying Prior Distributions |
|
|
550 | (11) |
|
15.1.1 Specification of Proficiency Variable Priors |
|
|
552 | (2) |
|
15.1.2 Specification of Evidence Model Priors |
|
|
554 | (6) |
|
15.1.3 Summary Statistics |
|
|
560 | (1) |
|
|
561 | (5) |
|
15.2.1 A Convenience Sample |
|
|
561 | (3) |
|
15.2.2 Item and other Exploratory Analyses |
|
|
564 | (2) |
|
15.3 Updating Based on Pilot Test Data |
|
|
566 | (13) |
|
15.3.1 Posterior Distributions |
|
|
566 | (9) |
|
15.3.2 Some Observations on Model Fit |
|
|
575 | (2) |
|
15.3.3 A Quick Validity Check |
|
|
577 | (2) |
|
|
579 | (4) |
|
16 The Future of Bayesian Networks in Educational Assessment |
|
|
583 | (18) |
|
16.1 Applications of Bayesian Networks |
|
|
583 | (3) |
|
16.2 Extensions to the Basic Bayesian Network Model |
|
|
586 | (7) |
|
16.2.1 Object-Oriented Bayes Nets |
|
|
586 | (2) |
|
16.2.2 Dynamic Bayesian Networks |
|
|
588 | (4) |
|
16.2.3 Assessment-Design Support |
|
|
592 | (1) |
|
16.3 Connections with Instruction |
|
|
593 | (3) |
|
16.3.1 Ubiquitous Assessment |
|
|
594 | (2) |
|
16.4 Evidence-Centered Assessment Design and Validity |
|
|
596 | (1) |
|
16.5 What We Still Do Not Know |
|
|
597 | (4) |
A Bayesian Network Resources |
|
601 | (6) |
|
|
601 | (3) |
|
A.1.1 Bayesian Network Manipulation |
|
|
602 | (1) |
|
A.1.2 Manual Construction of Bayesian Networks |
|
|
603 | (1) |
|
A.1.3 Markov Chain Monte Carlo |
|
|
603 | (1) |
|
A.2 Sample Bayesian Networks |
|
|
604 | (3) |
References |
|
607 | (32) |
Author Index |
|
639 | |