|
|
|
xv | |
|
|
|
xix | |
| Preface to Second Edition |
|
xxiii | |
| Preface to First Edition |
|
xxvii | |
| Acknowledgments |
|
xxxiii | |
|
Part I: Introduction to Meta-Analysis |
|
|
1 | (72) |
|
Integrating Research Findings Across Studies |
|
|
3 | (30) |
|
General Problem and an Example |
|
|
3 | (5) |
|
A Typical Interpretation of the Example Data |
|
|
4 | (3) |
|
Conclusions of the Review |
|
|
7 | (1) |
|
Critique of the Sample Review |
|
|
7 | (1) |
|
Problems With Statistical Significance Tests |
|
|
8 | (3) |
|
Is Statistical Power the Solution? |
|
|
11 | (2) |
|
|
|
13 | (2) |
|
|
|
15 | (2) |
|
Role of Meta-Analysis in the Behavioral and Social Sciences |
|
|
17 | (5) |
|
The Myth of the Perfect Study |
|
|
17 | (1) |
|
|
|
18 | (4) |
|
Role of Meta-Analysis in Theory Development |
|
|
22 | (2) |
|
Increasing Use of Meta-Analysis |
|
|
24 | (1) |
|
Meta-Analysis in Industrial-Organizational Psychology |
|
|
24 | (2) |
|
Wider Impact of Meta-Analysis on Psychology |
|
|
26 | (2) |
|
Impact of Meta-Analysis Outside Psychology |
|
|
28 | (1) |
|
|
|
28 | (1) |
|
Impact in Other Disciplines |
|
|
29 | (1) |
|
Meta-Analysis and Social Policy |
|
|
29 | (1) |
|
Meta-Analysis and Theories of Data |
|
|
30 | (2) |
|
|
|
32 | (1) |
|
Study Artifacts and Their Impact on Study Outcomes |
|
|
33 | (40) |
|
|
|
34 | (23) |
|
|
|
34 | (1) |
|
|
|
34 | (2) |
|
|
|
36 | (1) |
|
Range Variation in the Independent Variable |
|
|
37 | (2) |
|
Attrition Artifacts: Range Variation on the Dependent Variable |
|
|
39 | (2) |
|
Imperfect Construct Validity in the Independent Variable |
|
|
41 | (10) |
|
Imperfect Construct Validity in the Dependent Variable |
|
|
51 | (2) |
|
Computational and Other Errors in the Data |
|
|
53 | (1) |
|
Extraneous Factors Introduced by Study Procedure |
|
|
54 | (1) |
|
Bias in the Sample Correlation |
|
|
55 | (2) |
|
Sampling Error, Statistical Power, and the Interpretation of Research Findings |
|
|
57 | (8) |
|
An Illustration of Statistical Power |
|
|
57 | (2) |
|
A More Detailed Examination of Statistical Power |
|
|
59 | (6) |
|
|
|
65 | (1) |
|
Undercorrection for Artifacts in the Corrected Standard Deviation |
|
|
66 | (2) |
|
Coding Study Characteristics and Capitalization on Sampling Error in Moderator Analysis |
|
|
68 | (3) |
|
|
|
71 | (2) |
|
Part II: Meta-Analysis of Correlations |
|
|
73 | (168) |
|
Meta-Analysis of Correlations Corrected Individually for Artifacts |
|
|
75 | (62) |
|
Introduction and Overview |
|
|
75 | (6) |
|
Bare-Bones Meta-Analysis: Correcting for Sampling Error Only |
|
|
81 | (14) |
|
Estimation of Sampling Error |
|
|
81 | (2) |
|
Correcting the Variance for Sampling Error and a Worked Example |
|
|
83 | (7) |
|
Moderator Variables Analyzed by Grouping the Data and a Worked Example |
|
|
90 | (2) |
|
Correcting Feature Correlations for Sampling Error and a Worked Example |
|
|
92 | (3) |
|
Artifacts Other Than Sampling Error |
|
|
95 | (23) |
|
Error of Measurement and Correction for Attenuation |
|
|
95 | (8) |
|
Restriction or Enhancement of Range |
|
|
103 | (9) |
|
Dichotomization of Independent and Dependent Variables |
|
|
112 | (3) |
|
Imperfect Construct Validity in Independent and Dependent Variables |
|
|
115 | (2) |
|
|
|
117 | (1) |
|
|
|
117 | (1) |
|
|
|
118 | (1) |
|
Multiple Simultaneous Artifacts |
|
|
118 | (2) |
|
Meta-Analysis of Individually Corrected Correlations |
|
|
120 | (7) |
|
Individual Study Computations |
|
|
121 | (1) |
|
|
|
122 | (3) |
|
Final Meta-Analysis Estimation |
|
|
125 | (2) |
|
A Worked Example: Indirect Range Restriction |
|
|
127 | (5) |
|
Summary of Meta-Analysis Correcting Each Correlation Individually |
|
|
132 | (2) |
|
Exercise 1: Bare-Bones Meta-Analysis: Correcting for Sampling Error Only |
|
|
134 | (1) |
|
Exercise 2: Meta-Analysis Correcting Each Correlation Individually |
|
|
135 | (2) |
|
Meta-Analysis of Correlations Using Artifact Distributions |
|
|
137 | (52) |
|
Full Artifact Distribution Meta-Analysis |
|
|
138 | (31) |
|
|
|
140 | (2) |
|
The Standard Deviation of Correlations |
|
|
142 | (8) |
|
A Worked Example: Error of Measurement |
|
|
150 | (3) |
|
A Worked Example: Unreliability and Direct Range Restriction |
|
|
153 | (1) |
|
A Worked Example: Personnel Selection With Fixed Test (Direct Range Restriction) |
|
|
154 | (4) |
|
Personnel Selection With Varying Tests |
|
|
158 | (1) |
|
Personnel Selection: Findings and Formulas in the Literature |
|
|
159 | (7) |
|
A Worked Example: Indirect Range Restriction (Interactive Method) |
|
|
166 | (2) |
|
Refinements to Increase Accuracy of the SDp Estimate |
|
|
168 | (1) |
|
Accuracy of Corrections for Artifacts |
|
|
169 | (4) |
|
Mixed Meta-Analysis: Partial Artifact Information in Individual Studies |
|
|
173 | (7) |
|
An Example: Dichotomization of Both Variables |
|
|
175 | (5) |
|
Summary of Artifact Distribution Meta-Analysis of Correlations |
|
|
180 | (3) |
|
Phase 1: Cumulating Artifact Information |
|
|
181 | (1) |
|
Phase 2a: Correcting the Mean Correlation |
|
|
181 | (1) |
|
Phase 2b: Correcting the Standard Deviation of Correlations |
|
|
181 | (2) |
|
Exercise: Artifact Distribution Meta-Analysis |
|
|
183 | (6) |
|
Technical Questions in Meta-Analysis of Correlations |
|
|
189 | (52) |
|
r Versus r2: Which Should Be Used? |
|
|
189 | (3) |
|
r Versus Regression Slopes and Intercepts in Meta-Analysis |
|
|
192 | (3) |
|
|
|
192 | (1) |
|
|
|
192 | (1) |
|
Comparability of Units Across Studies |
|
|
193 | (1) |
|
Comparability of Findings Across Meta-Analyses |
|
|
194 | (1) |
|
Intrinsic Interpretability |
|
|
194 | (1) |
|
Technical Factors That Cause Overestimation of SDp |
|
|
195 | (6) |
|
Presence of Non-Pearson rs |
|
|
195 | (1) |
|
Presence of Outliers and Other Data Errors |
|
|
196 | (1) |
|
Use of r Instead of r in the Sampling Error Formula |
|
|
197 | (1) |
|
Undercorrection for Sampling Error Variance in the Presence of Range Restriction |
|
|
198 | (1) |
|
Nonlinearity in the Range Correction |
|
|
198 | (2) |
|
Other Factors Causing Overestimation of SDp |
|
|
200 | (1) |
|
Fixed- and Random-Effects Models in Meta-Analysis |
|
|
201 | (4) |
|
Accuracy of Different Random-Effects Models |
|
|
203 | (2) |
|
Credibility Versus Confidence Intervals in Meta-Analysis |
|
|
205 | (1) |
|
Computing Confidence Intervals in Meta-Analysis |
|
|
206 | (1) |
|
Range Restriction in Meta-Analysis: New Technical Analysis |
|
|
207 | (1) |
|
Domains With No Range Restriction |
|
|
208 | (5) |
|
|
|
209 | (1) |
|
Systematic Error of Measurement |
|
|
210 | (1) |
|
Artificial Dichotomization |
|
|
210 | (1) |
|
|
|
211 | (1) |
|
Meta-Analysis for Simple Artifacts |
|
|
211 | (2) |
|
|
|
213 | (11) |
|
Range Restriction as a Single Artifact |
|
|
213 | (2) |
|
Correction for Direct Range Restriction |
|
|
215 | (1) |
|
Meta-Analysis for Range Restriction as a Single Artifact |
|
|
215 | (1) |
|
Two Populations in Direct Range Restriction |
|
|
216 | (1) |
|
Error of Measurement in the Independent Variable in Direct Range Restriction |
|
|
216 | (3) |
|
Error of Measurement in the Dependent Variable in Direct Range Restriction |
|
|
219 | (2) |
|
Error of Measurement in Both Variables: Direct Range Restriction |
|
|
221 | (1) |
|
Meta-Analysis in Direct Range Restriction: Previous Work |
|
|
221 | (1) |
|
Educational and Employment Selection |
|
|
222 | (1) |
|
Meta-Analysis Correcting Correlations Individually: Direct Range Restriction |
|
|
223 | (1) |
|
Artifact Distribution Meta-Analysis: Direct Range Restriction |
|
|
224 | (1) |
|
Indirect Range Restriction |
|
|
224 | (16) |
|
A Causal Model for Indirect Range Restriction |
|
|
226 | (2) |
|
|
|
228 | (1) |
|
Range Restriction on Other Variables in Indirect Range Restriction |
|
|
228 | (1) |
|
Estimation in Indirect Range Restriction |
|
|
229 | (1) |
|
The Correlation Between S and T in Indirect Range Restriction |
|
|
230 | (1) |
|
The Attenuation Model in Indirect Range Restriction |
|
|
231 | (1) |
|
Predictor Measurement Error in Indirect Range Restriction |
|
|
232 | (1) |
|
Meta-Analysis Correcting Each Correlation Individually: Indirect Range Restriction |
|
|
233 | (1) |
|
Artifact Distribution Meta-Analysis for Indirect Range Restriction |
|
|
233 | (7) |
|
Criticisms of Meta-Analysis Procedures for Correlations |
|
|
240 | (1) |
|
Part III: Meta-Analysis of Experimental Effects and Other Dichotomous Comparisons |
|
|
241 | (150) |
|
Treatment Effects: Experimental Artifacts and Their Impact |
|
|
243 | (30) |
|
Quantification of the Treatment Effect: The d Statistic and the Point Biserial Correlation |
|
|
244 | (3) |
|
Sampling Error in d Values: Illustrations |
|
|
247 | (5) |
|
|
|
248 | (2) |
|
|
|
250 | (1) |
|
|
|
251 | (1) |
|
Error of Measurement in the Dependent Variable |
|
|
252 | (4) |
|
Error of Measurement in the Treatment Variable |
|
|
256 | (4) |
|
Variation Across Studies in Treatment Strength |
|
|
260 | (1) |
|
Range Variation on the Dependent Variable |
|
|
261 | (1) |
|
Dichotomization of the Dependent Variable |
|
|
262 | (2) |
|
Imperfect Construct Validity in the Dependent Variable |
|
|
264 | (2) |
|
Imperfect Construct Validity in the Treatment Variable |
|
|
266 | (1) |
|
Bias in the Effect Size (d Statistic) |
|
|
266 | (2) |
|
Recording, Computational, and Transcriptional Errors |
|
|
268 | (1) |
|
Multiple Artifacts and Corrections |
|
|
269 | (4) |
|
Meta-Analysis Methods for d Values |
|
|
273 | (62) |
|
Effect Size Indexes: d and r |
|
|
275 | (7) |
|
Maximum Value of Point Biserial r |
|
|
276 | (1) |
|
The Effect Size (d Statistic) |
|
|
277 | (2) |
|
Correction of the Point Biserial r for Unequal Sample Sizes |
|
|
279 | (1) |
|
Examples of the Convertibility of r and d |
|
|
280 | (2) |
|
Problems of Artificial Dichotomization |
|
|
282 | (1) |
|
An Alternative to d: Glass's d |
|
|
282 | (1) |
|
Sampling Error in the d Statistic |
|
|
283 | (3) |
|
|
|
283 | (3) |
|
The Confidence Interval for δ |
|
|
286 | (1) |
|
Cumulation and Correction of the Variance for Sampling Error |
|
|
286 | (6) |
|
|
|
287 | (2) |
|
A Worked Numerical Example |
|
|
289 | (2) |
|
Another Example: Leadership Training by Experts |
|
|
291 | (1) |
|
Analysis of Moderator Variables |
|
|
292 | (9) |
|
Using Study Domain Subsets |
|
|
293 | (1) |
|
Using Study Characteristic Correlations |
|
|
294 | (1) |
|
A Worked Example: Training by Experts Versus Training by Managers |
|
|
295 | (3) |
|
Another Worked Example: Amount of Training |
|
|
298 | (3) |
|
The Correlational Moderator Analysis |
|
|
301 | (1) |
|
Correcting d-Value Statistics for Measurement Error in the Dependent Variable |
|
|
301 | (12) |
|
Meta-Analysis of d Values Corrected Individually and a Worked Example |
|
|
305 | (3) |
|
Artifact Distribution Meta-Analysis and a Worked Example |
|
|
308 | (5) |
|
Measurement Error in the Independent Variable in Experiments |
|
|
313 | (2) |
|
Other Artifacts and Their Effects |
|
|
315 | (1) |
|
Correcting for Multiple Artifacts |
|
|
316 | (12) |
|
Attenuation Effect of Multiple Artifacts and Correction for the Same |
|
|
317 | (2) |
|
Disattenuation and Sampling Error: The Confidence Interval |
|
|
319 | (1) |
|
A Formula for Meta-Analysis With Multiple Artifacts |
|
|
320 | (8) |
|
Summary of Meta-Analysis of d Values |
|
|
328 | (3) |
|
Exercise: Meta-Analysis of d Values |
|
|
331 | (4) |
|
Technical Questions in Meta-Analysis of d Values |
|
|
335 | (56) |
|
Alternative Experimental Designs |
|
|
335 | (2) |
|
Within-Subjects Experimental Designs |
|
|
337 | (33) |
|
The Potentially Perfect Power of the Pre-Post Design |
|
|
338 | (1) |
|
Deficiencies of the Between-Subjects Design |
|
|
339 | (5) |
|
Error of Measurement and the Within-Subjects Design |
|
|
344 | (5) |
|
The Treatment by Subjects Interaction |
|
|
349 | (7) |
|
|
|
356 | (14) |
|
Meta-Analysis and the Within-Subjects Design |
|
|
370 | (4) |
|
|
|
370 | (1) |
|
The Treatment by Subjects Interaction |
|
|
371 | (3) |
|
Statistical Power in the Two Designs |
|
|
374 | (8) |
|
Designs Matched for Number of Subjects |
|
|
375 | (3) |
|
Designs Matched for Number of Measurements or Scores |
|
|
378 | (4) |
|
Threats to Internal and External Validity |
|
|
382 | (5) |
|
|
|
383 | (1) |
|
|
|
384 | (1) |
|
|
|
384 | (1) |
|
|
|
384 | (1) |
|
|
|
385 | (1) |
|
|
|
386 | (1) |
|
Interaction Between Testing and Treatment |
|
|
387 | (1) |
|
Interaction Between Selection and Treatment |
|
|
387 | (1) |
|
Bias in Observed d Values |
|
|
387 | (1) |
|
Use of Multiple Regression in Moderator Analysis of d Values |
|
|
388 | (3) |
|
Part IV: General Issues in Meta-Analysis |
|
|
391 | (126) |
|
General Technical Issues in Meta-Analysis |
|
|
393 | (36) |
|
Fixed-Effects Versus Random-Effects Models in Meta-Analysis |
|
|
393 | (6) |
|
Second-Order Sampling Error: General Principles |
|
|
399 | (2) |
|
Detecting Moderators Not Hypothesized a Priori |
|
|
401 | (5) |
|
Second-Order Meta-Analyses |
|
|
406 | (2) |
|
Large-N Studies and Meta-Analysis |
|
|
408 | (3) |
|
Second-Order Sampling Error: Technical Treatment |
|
|
411 | (12) |
|
|
|
415 | (2) |
|
|
|
417 | (1) |
|
|
|
418 | (2) |
|
The Leadership Training by Experts Example |
|
|
420 | (1) |
|
The Skills Training Moderator Example |
|
|
421 | (2) |
|
The Detection of Moderator Variables: Summary |
|
|
423 | (1) |
|
Hierarchical Analysis of Moderator Variables |
|
|
424 | (3) |
|
Exercise: Second-Order Meta-Analysis |
|
|
427 | (2) |
|
Cumulation of Findings Within Studies |
|
|
429 | (16) |
|
|
|
429 | (1) |
|
|
|
430 | (2) |
|
Conceptual Replication and Confirmatory Factor Analysis |
|
|
432 | (3) |
|
Conceptual Replication: An Alternative Approach |
|
|
435 | (4) |
|
|
|
439 | (3) |
|
Subgroups and Loss of Power |
|
|
440 | (1) |
|
Subgroups and Capitalization on Chance |
|
|
440 | (1) |
|
Subgroups and Suppression of Data |
|
|
441 | (1) |
|
Subgroups and the Bias of Disaggregation |
|
|
441 | (1) |
|
Conclusion: Use Total Group Correlations |
|
|
442 | (1) |
|
|
|
442 | (3) |
|
Methods of Integrating Findings Across Studies and Related Software |
|
|
445 | (22) |
|
The Traditional Narrative Procedure |
|
|
445 | (1) |
|
The Traditional Voting Method |
|
|
446 | (1) |
|
Cumulation of p Values Across Studies |
|
|
447 | (2) |
|
Statistically Correct Vote-Counting Procedures |
|
|
449 | (4) |
|
Vote-Counting Methods Yielding Only Significance Levels |
|
|
449 | (1) |
|
Vote-Counting Methods Yielding Estimates of Effect Sizes |
|
|
450 | (3) |
|
Meta-Analysis of Research Studies |
|
|
453 | (10) |
|
Descriptive Meta-Analysis Methods: Glassian and Related Methods |
|
|
454 | (4) |
|
Meta-Analysis Methods Focusing Only on Sampling Error: Hedges's Methods, Rosenthal's Methods, and Bare-Bones Methods |
|
|
458 | (3) |
|
Psychometric Meta-Analysis: Correction for Multiple Artifacts |
|
|
461 | (2) |
|
Unresolved Problems in Meta-Analysis |
|
|
463 | (1) |
|
Summary of Methods of Integrating Studies |
|
|
463 | (1) |
|
Computer Programs for Meta-Analysis |
|
|
464 | (3) |
|
Locating, Evaluating, Selecting, and Coding Studies |
|
|
467 | (26) |
|
Conducting a Thorough Literature Search |
|
|
467 | (1) |
|
What to Do About Studies With Methodological Weaknesses |
|
|
468 | (2) |
|
Coding Studies in Meta-Analysis |
|
|
470 | (1) |
|
What to Include in the Meta-Analysis Report |
|
|
471 | (2) |
|
Information Needed in Reports of Primary Studies |
|
|
473 | (6) |
|
|
|
473 | (1) |
|
|
|
474 | (1) |
|
Studies Using Multiple Regression |
|
|
475 | (1) |
|
Studies Using Factor Analysis |
|
|
476 | (1) |
|
Studies Using Canonical Correlation |
|
|
476 | (1) |
|
Studies Using Multivariate Analysis of Variance (MANOVA) |
|
|
477 | (1) |
|
General Comments on Reporting in Primary Studies |
|
|
477 | (2) |
|
Appendix: Coding Sheet for Validity Studies |
|
|
479 | (14) |
|
Availability and Source Bias in Meta-Analysis |
|
|
493 | (18) |
|
Some Evidence on Publication Bias |
|
|
494 | (1) |
|
Effects of Methodological Quality on Mean Effect Sizes From Different Sources |
|
|
495 | (1) |
|
Multiple Hypotheses and Other Considerations in Availability Bias |
|
|
496 | (2) |
|
Methods for Detecting Availability Bias |
|
|
498 | (5) |
|
File Drawer Analysis Based on p Values |
|
|
499 | (1) |
|
File Drawer Analysis Based on Effect Size |
|
|
500 | (1) |
|
A Graphic Method for Detecting Availability Bias: The Funnel Plot |
|
|
501 | (2) |
|
Methods for Correcting for Availability Bias |
|
|
503 | (8) |
|
The Original Hedges-Olkin (1985) Method |
|
|
504 | (1) |
|
The Iyengar-Greenhouse (1988) Method |
|
|
505 | (1) |
|
The Begg-Mazumdar (1994) Method |
|
|
505 | (1) |
|
Further Work by Hedges and Associates |
|
|
506 | (2) |
|
The Duval-Tweedie (2000) Trim-and-Fill Method |
|
|
508 | (1) |
|
Summary of Methods for Correcting Availability Bias |
|
|
509 | (2) |
|
Summary of Psychometric Meta-Analysis |
|
|
511 | (6) |
|
Meta-Analysis Methods and Theories of Data |
|
|
511 | (1) |
|
What Is the Ultimate Purpose of Meta-Analysis? |
|
|
512 | (1) |
|
Psychometric Meta-Analysis: Summary Overview |
|
|
513 | (4) |
| Appendix: Windows-Based Meta-Analysis Software Package |
|
517 | (10) |
| References |
|
527 | (36) |
| Name Index |
|
563 | (6) |
| Subject Index |
|
569 | (12) |
| About the Authors |
|
581 | |
