About the Online Companion Materials |
|
xi | |
About the Authors |
|
xiii | |
Preface |
|
xv | |
|
|
xv | |
|
Topics and Corresponding Chapters |
|
|
xv | |
|
The Book's Iterative Teaching Style |
|
|
xvi | |
|
Using the Online Companion Materials |
|
|
xvii | |
Acknowledgments |
|
xix | |
Section I Foundational Topics |
|
|
Chapter 1 Introduction to Systematic Observation and Measurement Contexts |
|
|
3 | (20) |
|
Systematic Observation Using Count Coding |
|
|
3 | (6) |
|
Alternatives to Systematic Observation |
|
|
4 | (1) |
|
Ways to Quantify Observations |
|
|
4 | (2) |
|
The Rationale for Systematic Observation Using Count Coding |
|
|
6 | (3) |
|
The Importance of Falsifiable Research Questions or Hypotheses |
|
|
9 | (1) |
|
Objects of Measurement: The Continuum of Context- Dependent Behaviors to Generalized Person Characteristics |
|
|
10 | (5) |
|
Context-Dependent Behaviors |
|
|
10 | (1) |
|
Generalized Person Characteristics |
|
|
11 | (4) |
|
Generalized Behavioral Tendencies |
|
|
12 | (1) |
|
|
13 | (2) |
|
Judging the Relative Scientific Value of Different Measures |
|
|
15 | (5) |
|
|
15 | (1) |
|
|
16 | (1) |
|
|
17 | (1) |
|
|
18 | (2) |
|
Conclusions and Recommendations |
|
|
20 | (3) |
|
Chapter 2 Validation of Observational Variables |
|
|
23 | (22) |
|
The Changing Concept of Validation |
|
|
24 | (1) |
|
Consequences of Not Attending to Validation |
|
|
25 | (1) |
|
Overview: Types of Validity by Objects of Measurement and Purposes |
|
|
26 | (1) |
|
|
27 | (1) |
|
Varying Importance Ascribed to Content Validation |
|
|
27 | (1) |
|
Weaknesses of Content Validation |
|
|
28 | (1) |
|
|
28 | (3) |
|
Influences on Sensitivity to Change |
|
|
29 | (1) |
|
Weakness of Sensitivity to Change as Way to Judge a Variable's Validity |
|
|
30 | (1) |
|
Criterion-Related Validation |
|
|
31 | (1) |
|
The Primary Appeal of Criterion-Related Validation |
|
|
31 | (1) |
|
Weaknesses of Criterion-Related Validation |
|
|
32 | (1) |
|
|
32 | (6) |
|
|
33 | (1) |
|
Discriminative Validation Evidence |
|
|
33 | (1) |
|
Nomological Validation Evidence |
|
|
34 | (1) |
|
Weaknesses of Convergent Validity |
|
|
34 | (1) |
|
Methods That Combine Convergent and Divergent Validity |
|
|
34 | (1) |
|
Multitrait, Multimethod (MTMM) Validation |
|
|
35 | (1) |
|
Confirmatory Factor Analysis as a Method of Validation |
|
|
35 | (3) |
|
Putting It All Together With Literature Synthesis |
|
|
38 | (1) |
|
An Implicit Weakness of Science? |
|
|
39 | (2) |
|
Conclusions and Recommendations |
|
|
41 | (4) |
|
Chapter 3 Estimating Stable Measures of Generalized Person Characteristics Through Systematic Observation |
|
|
45 | (16) |
|
A Brief Overview of Measurement Theory |
|
|
46 | (1) |
|
Why Stable Estimates Maximize Convergent Construct Validity |
|
|
46 | (2) |
|
Two Ways to Stabilize Observational Measures |
|
|
48 | (9) |
|
Estimating Stable Skills Through Observation |
|
|
49 | (1) |
|
Definition of Measurement Context |
|
|
49 | (1) |
|
How Controlling Influential Contextual Variables Stabilizes Skill Estimates |
|
|
50 | (1) |
|
Why Skills Are Often Assessed in Clinics or Labs |
|
|
50 | (2) |
|
Estimating Stable Generalized Behavioral Tendencies Through Observation |
|
|
52 | (10) |
|
Representativeness, Revisited |
|
|
52 | (1) |
|
Definition of Contextual Measurement Error |
|
|
53 | (1) |
|
Contextual Measurement Error in Measures of Generalized Behavioral Tendencies |
|
|
54 | (1) |
|
How Averaging Scores Across Contexts Improves Measures of Generalized Behavioral Tendencies |
|
|
55 | (2) |
|
Naturalness of Observations and Representativeness, Revisited |
|
|
57 | (1) |
|
Computing Stability Coefficients |
|
|
57 | (2) |
|
Conclusions and Recommendations |
|
|
59 | (2) |
|
Chapter 4 Designing or Adapting Coding Manuals |
|
|
61 | (18) |
|
Definition of a Coding Manual |
|
|
61 | (1) |
|
Deciding Whether to Write a New Coding Manual |
|
|
62 | (1) |
|
Recommended Steps for Modifying or Designing Coding Manuals |
|
|
62 | (13) |
|
Define Start and Stop Coding Rules |
|
|
62 | (2) |
|
Conceptually Define the Object of Measurement |
|
|
64 | (1) |
|
Define the Highest Level of Codable Behavior |
|
|
64 | (1) |
|
Determine the Level of Distinction Coders Have to Make |
|
|
65 | (2) |
|
Organize the Coded Categories into Mutually Exclusive Sets |
|
|
67 | (1) |
|
Decide How to Use Physically Based and/or Socially Based Definitions |
|
|
68 | (1) |
|
Define the Lowest-Level Categories |
|
|
69 | (2) |
|
Determine Sources of Conceptual and Operational Definitions |
|
|
71 | (3) |
|
|
74 | (1) |
|
The Potential Value of Flowcharts |
|
|
75 | (1) |
|
Recommended Length of Coding Manuals |
|
|
75 | (1) |
|
Conclusions and Recommendations |
|
|
76 | (3) |
|
|
79 | (20) |
|
The Elements of an Observational Measurement System |
|
|
79 | (1) |
|
|
80 | (9) |
|
The Superordinate Distinctions: Continuous Versus Intermittent |
|
|
80 | (1) |
|
The Subordinate Distinctions: Continuous Versus Intermittent |
|
|
81 | (2) |
|
|
81 | (1) |
|
|
82 | (1) |
|
|
82 | (1) |
|
Types of Interval Sampling |
|
|
83 | (5) |
|
|
84 | (1) |
|
Momentary-Interval Sampling |
|
|
84 | (1) |
|
Partial-Interval Sampling |
|
|
85 | (3) |
|
Summary of Interval Sampling |
|
|
88 | (1) |
|
Which Dimension of Behavior Should Be Estimated |
|
|
88 | (1) |
|
Summary of Behavior Sampling |
|
|
88 | (1) |
|
|
89 | (1) |
|
|
90 | (1) |
|
|
90 | (1) |
|
|
90 | (1) |
|
|
90 | (2) |
|
When to Code Relative to When the Behavior Occurs |
|
|
92 | (2) |
|
|
92 | (1) |
|
Coding From Recorded Sessions |
|
|
93 | (1) |
|
Recording Coding Decisions |
|
|
94 | (1) |
|
|
94 | (1) |
|
|
95 | (1) |
|
Conclusions and Recommendations |
|
|
95 | (4) |
|
Chapter 6 Common Metrics of Observational Variables |
|
|
99 | (18) |
|
|
100 | (1) |
|
Quantifiable Dimensions of Behavior |
|
|
100 | (1) |
|
|
101 | (6) |
|
How Proportion Metrics Change the Meaning of Observational Variables |
|
|
102 | (1) |
|
|
103 | (1) |
|
An Implicit Assumption of Proportion Metrics |
|
|
104 | (1) |
|
Testing Whether the Data Fit the Assumption of Proportion Metrics |
|
|
105 | (1) |
|
Consequences of Using a Proportion When the Data Do Not Fit the Assumption |
|
|
106 | (1) |
|
Alternative Methods to Control Influential Contextual Variables |
|
|
107 | (1) |
|
|
107 | (1) |
|
|
108 | (1) |
|
Aggregate Measures of Generalized Person Characteristics |
|
|
108 | (3) |
|
|
110 | (1) |
|
|
110 | (1) |
|
Group Analysis of Observational Variables |
|
|
111 | (3) |
|
|
112 | (1) |
|
|
112 | (2) |
|
Analyzing Count Variables |
|
|
114 | (1) |
|
Conclusions and Recommendations |
|
|
114 | (3) |
|
Chapter 7 Training Observers and Preventing Observer Drift |
|
|
117 | (20) |
|
Point-by-Point Agreement and Disagreement |
|
|
118 | (9) |
|
Point-by-Point Agreement of Interval-Sampled Data |
|
|
118 | (2) |
|
Point-by-Point Agreement of Timed-Event Data |
|
|
120 | (4) |
|
|
124 | (3) |
|
|
127 | (7) |
|
Using Discrepancy Discussions to Train Observers |
|
|
129 | (3) |
|
Creating Criterion-Coding Standards |
|
|
130 | (1) |
|
Training Observers: Remaining Steps |
|
|
131 | (1) |
|
Preventing Observer Drift |
|
|
132 | (8) |
|
Choosing a Method of Selecting Sessions for Agreement Checks |
|
|
132 | (1) |
|
Preventing or Addressing Observer Drift: Remaining Steps |
|
|
133 | (1) |
|
Conclusions and Recommendations |
|
|
134 | (3) |
|
Chapter 8 Interobserver Reliability of Observational Variables |
|
|
137 | (24) |
|
General Principles of Interobserver Reliability Estimation |
|
|
138 | (2) |
|
Single-Case Design Concepts of Interobserver Reliability |
|
|
140 | (9) |
|
Session-Level Agreement Indices |
|
|
142 | (7) |
|
|
142 | (1) |
|
|
142 | (5) |
|
Base Rate and All Indices of Point-by-Point Agreement |
|
|
147 | (1) |
|
Summary of Point-by-Point Agreement Indices |
|
|
147 | (2) |
|
Group-Design Concepts of Interobserver Reliability |
|
|
149 | (4) |
|
A Sample-Level Reliability Index: Intraclass Correlation |
|
|
149 | (1) |
|
Why Session-Level Reliability Is Insufficient for Group-Design Studies |
|
|
150 | (2) |
|
The Interpretation of IBM SPSS Software Output for ICC |
|
|
152 | (1) |
|
The Relation Between Interobserver Agreement and ICC |
|
|
153 | (1) |
|
The Special Case of Fidelity of Treatment Data |
|
|
153 | (1) |
|
Selection of Interobserver Reliability Index |
|
|
154 | (1) |
|
Consequences of Low or Unknown Interobserver Reliability |
|
|
154 | (2) |
|
Conclusions and Recommendations |
|
|
156 | (5) |
Section II Advanced Topics |
|
|
Chapter 9 Introduction to Sequential Analysis |
|
|
161 | (20) |
|
About the Terminology Used in This Chapter |
|
|
162 | (1) |
|
Sequential Versus Nonsequential Variable Metrics |
|
|
162 | (1) |
|
Requirements for Sequential Analysis |
|
|
163 | (1) |
|
Why Sequential Associations Are Insufficient for Causal Inferences |
|
|
164 | (1) |
|
Coded Units and Contingency Tables |
|
|
164 | (1) |
|
Four Types of Sequential Analysis |
|
|
165 | (8) |
|
|
166 | (1) |
|
|
167 | (3) |
|
|
170 | (1) |
|
|
171 | (2) |
|
Observational Software for Sequential Analysis |
|
|
173 | (1) |
|
The Need to Control for Chance |
|
|
173 | (1) |
|
Indices of Sequential Association |
|
|
174 | (4) |
|
Transitional Probabilities |
|
|
175 | (1) |
|
|
175 | (1) |
|
|
176 | (1) |
|
Relative Advantages and Disadvantages Across Indices |
|
|
177 | (1) |
|
Conclusions and Recommendations |
|
|
178 | (3) |
|
Chapter 10 Research Questions Involving Sequential Associations |
|
|
181 | (20) |
|
Sequential Analysis in Within-Group and Between-Groups Designs |
|
|
181 | (3) |
|
Testing the Significance of Mean Sequential Associations |
|
|
183 | (1) |
|
Testing Between-Groups Differences in Mean Sequential Associations |
|
|
183 | (1) |
|
Testing Within-Group Differences in Mean Sequential Associations |
|
|
183 | (1) |
|
Testing Summary-Level Associations Between Participant Characteristics and Sequential Associations |
|
|
184 | (1) |
|
Sequential Analysis in Single-Case Designs |
|
|
184 | (10) |
|
The Meaning of Contingency in Behavior Analysis |
|
|
185 | (1) |
|
Why Significance Testing Is Controversial at the Individual Participant Level |
|
|
186 | (1) |
|
Types of Within-Participant Research Questions and Methods to Address Them |
|
|
187 | (7) |
|
Descriptive Questions to Inform or Supplement Single-Case Experiments |
|
|
187 | (1) |
|
Transitional Probability Comparisons and Contingency Space Analysis |
|
|
188 | (2) |
|
Contingency Indices as Dependent Variables in Single-Case Experimental Designs |
|
|
190 | (2) |
|
Contingency Indices as Procedural Fidelity Measures in Single-Case Experimental Designs |
|
|
192 | (2) |
|
Data Sufficiency for Sequential Analysis |
|
|
194 | (2) |
|
Consequences of Insufficient Data |
|
|
195 | (1) |
|
Defining Sufficient Data for Estimating Sequential Associations |
|
|
195 | (1) |
|
Proposed Solutions When Data Are Insufficient |
|
|
196 | (1) |
|
Conclusions and Recommendations |
|
|
197 | (4) |
|
Chapter 11 Generalizability Theory |
|
|
201 | (16) |
|
|
201 | (1) |
|
Overview of G Theory and Definition of Terms |
|
|
202 | (2) |
|
A Sample Observer-by-Context G and D Study |
|
|
204 | (5) |
|
The Rationale for Preferring the Absolute G Coefficient |
|
|
209 | (1) |
|
Sample Applications of D Studies |
|
|
209 | (1) |
|
|
210 | (2) |
|
Conclusions and Recommendations |
|
|
212 | (5) |
Section III Putting It All Together |
|
|
Chapter 12 Summary of Recommendations for Best Practices in Observational Measurement |
|
|
217 | (12) |
|
Identify Research Questions and Objects of Measurement |
|
|
217 | (1) |
|
Validate Observational Variables |
|
|
218 | (2) |
|
Design or Adapt Coding Manuals |
|
|
220 | (1) |
|
Select Each Component of the Coding Enterprise |
|
|
220 | (2) |
|
Select Observational Variable Metrics |
|
|
222 | (1) |
|
|
223 | (1) |
|
Prevent, Detect, and Address Observer Drift |
|
|
224 | (1) |
|
Estimate, Report, and Interpret Interobserver Reliability |
|
|
225 | (2) |
|
Use Sequential Analysis to Address Research Questions Involving Sequential Associations or Contingencies |
|
|
227 | (1) |
|
Apply Generalizability Theory to Improve Reliability of Observational Measures of Generalized Person Characteristics |
|
|
227 | (2) |
Glossary |
|
229 | (12) |
Index |
|
241 | |