|
List of Figures and Tables |
|
|
xiii | |
Preface |
|
xvii | |
|
The Evaluation Research and Evidence-Based Practice Partnership |
|
|
1 | (32) |
|
|
2 | (1) |
|
Evaluation Research: What It Is and What It Is Not |
|
|
2 | (14) |
|
So the Evidence Is Valid, but Does It Matter? |
|
|
16 | (2) |
|
Evidence-Based Medicine (EBM): Some History and Definitions |
|
|
18 | (5) |
|
|
18 | (2) |
|
The Definitions: Research Evidence, Clinical Expertise, and Patient Values |
|
|
20 | (3) |
|
Cietting It Done: The Steps to EBM |
|
|
23 | (2) |
|
EBM and EBP: Perfect or Not So? |
|
|
25 | (1) |
|
Getting It Together: Evaluation Research, EBM/EBP, and the Research Consumer |
|
|
25 | (2) |
|
Summary of Chapter 1: Evaluation Research and Evidence-Based Practice Partnership |
|
|
27 | (1) |
|
|
27 | (1) |
|
|
28 | (1) |
|
|
29 | (3) |
|
|
32 | (1) |
|
The Research Consumer as Detective: Investigating Program and Bibliographic Databases |
|
|
33 | (32) |
|
Looking in All the Right Places: Finding the Evidence |
|
|
34 | (2) |
|
|
36 | (4) |
|
Other Places, Other Programs: Expanding the Search |
|
|
40 | (3) |
|
The Literature: The Research Consumer's Support |
|
|
43 | (1) |
|
The Consumer as Reviewer: Eight Literature Reviewing Tasks |
|
|
44 | (1) |
|
Choosing an Online Bibliographic Database |
|
|
45 | (2) |
|
A Note on Online or Electronic Journals |
|
|
47 | (1) |
|
|
47 | (3) |
|
Research Questions and Descriptors and Key Words |
|
|
50 | (2) |
|
Even More Search Terms: Authors, Titles, Title Words, and Journals and Then Some---Limiting the Search |
|
|
52 | (1) |
|
How Do You Ask for Information? Searching With Boolean Operators |
|
|
53 | (2) |
|
Pausing During the Search |
|
|
55 | (1) |
|
Supplementing the Online Search |
|
|
55 | (1) |
|
|
56 | (1) |
|
Is Everything Worthwhile Published? |
|
|
57 | (1) |
|
|
58 | (2) |
|
Summary of Chapter 2: The Research Consumer as Detective |
|
|
60 | (1) |
|
|
60 | (1) |
|
|
61 | (1) |
|
|
62 | (2) |
|
|
64 | (1) |
|
The Practical Research Consumer |
|
|
65 | (26) |
|
The Research Consumer's Practical Side |
|
|
67 | (1) |
|
The Practical Screen, Part 1: Language, Design, the Program, Timeliness, and Sponsorship |
|
|
67 | (6) |
|
The Practical Screen, Part 2: Outcomes and Participants |
|
|
73 | (1) |
|
Effectiveness Is a Comparative Concern: Outcomes |
|
|
73 | (4) |
|
|
77 | (3) |
|
|
80 | (1) |
|
Program Activities and Resources |
|
|
81 | (1) |
|
Evaluation Report or Something Else? |
|
|
81 | (3) |
|
Are Evaluation Reports All There Is? |
|
|
84 | (1) |
|
Summary of Chapter 3: The Practical Research Consumer |
|
|
85 | (1) |
|
|
85 | (1) |
|
|
86 | (1) |
|
|
87 | (4) |
|
The Designing Research Consumer |
|
|
91 | (54) |
|
Research Design: Creating the Structure |
|
|
92 | (2) |
|
The Randomized Controlled Trial: Going for the Gold |
|
|
94 | (18) |
|
Randomized Controlled Trial or True Experiment: Variations on a Theme |
|
|
96 | (1) |
|
Concurrent Controls: Do It at the Same Time |
|
|
96 | (1) |
|
Wait-List Control: Do It Sequentially |
|
|
97 | (7) |
|
|
104 | (1) |
|
|
105 | (2) |
|
|
107 | (2) |
|
|
109 | (3) |
|
Quasi-Experimental Evaluation Designs |
|
|
112 | (6) |
|
Nonrandomized Controlled Trials: Concurrent Controls |
|
|
112 | (3) |
|
|
115 | (1) |
|
Self-Controlled or Pretest-Posttest Designs |
|
|
116 | (1) |
|
|
117 | (1) |
|
Interrupted or Single Time-Series Designs |
|
|
117 | (1) |
|
|
118 | (9) |
|
|
119 | (1) |
|
Prospective Cohort Design |
|
|
119 | (2) |
|
|
121 | (2) |
|
|
123 | (4) |
|
Observational Designs and Controlled Trials |
|
|
127 | (1) |
|
The Bottom Line: Internal and External Validity |
|
|
128 | (1) |
|
Internal Validity Is Threatened |
|
|
128 | (2) |
|
External Validity Is Threatened |
|
|
130 | (9) |
|
The Problem of Incomparable Participants: Statistical Methods to the Rescue |
|
|
134 | (1) |
|
|
135 | (1) |
|
|
136 | (3) |
|
Summary of Chapter 4: The Designing Research Consumer |
|
|
139 | (1) |
|
|
139 | (1) |
|
|
140 | (1) |
|
|
141 | (4) |
|
The Research Consumer Reviews the Measures |
|
|
145 | (40) |
|
Evaluation's Main Measures |
|
|
146 | (3) |
|
Gathering the Data: Weighing the Options |
|
|
149 | (21) |
|
Self-Administered Surveys |
|
|
149 | (3) |
|
Advantages of Self-Administered Surveys (Questionnaires) |
|
|
152 | (1) |
|
|
152 | (1) |
|
Forced-Choice Achievement Tests |
|
|
153 | (1) |
|
Advantages of Achievement Tests |
|
|
154 | (1) |
|
|
154 | (1) |
|
|
155 | (2) |
|
|
157 | (1) |
|
|
157 | (1) |
|
|
158 | (2) |
|
Advantages of Observations |
|
|
160 | (1) |
|
|
161 | (1) |
|
|
161 | (2) |
|
|
163 | (1) |
|
|
163 | (1) |
|
Large Databases and Data Sets From Previous Studies |
|
|
164 | (3) |
|
Advantages of Existing Databases and Data Sets |
|
|
167 | (1) |
|
|
167 | (2) |
|
|
169 | (1) |
|
|
169 | (1) |
|
|
169 | (1) |
|
|
170 | (1) |
|
When One Measure Is Not Sufficient Measurement |
|
|
170 | (3) |
|
Responses and Ratings: Choices for Participants and Evaluators |
|
|
173 | (1) |
|
|
174 | (1) |
|
Not Quite Free to Choose: Data Collection With Preset Choices |
|
|
175 | (2) |
|
Response Choices: Getting the Data From the Questions |
|
|
177 | (5) |
|
Summary of Chapter 5: The Research Consumer Reviews the Measures |
|
|
182 | (1) |
|
|
182 | (1) |
|
|
183 | (1) |
|
|
184 | (1) |
|
The Research Consumer Evaluates Measurement Reliability and Validity |
|
|
185 | (28) |
|
Reliability and Validity: A Team Approach |
|
|
187 | (1) |
|
|
188 | (7) |
|
Within Measure Reliability |
|
|
189 | (1) |
|
|
189 | (1) |
|
Internal Consistency Reliability |
|
|
190 | (2) |
|
|
192 | (1) |
|
Between Measures Reliability |
|
|
193 | (1) |
|
Intra- and Inter-Rater Reliability |
|
|
193 | (2) |
|
|
195 | (3) |
|
|
195 | (1) |
|
|
196 | (1) |
|
|
196 | (1) |
|
|
196 | (2) |
|
Sensitivity and Specificity |
|
|
198 | (1) |
|
Needing It All or Just Needing Some of It: Reliability, Validity, Sensitivity, and Specificity |
|
|
198 | (2) |
|
|
200 | (1) |
|
Reliability and Validity Within and Across Evaluations |
|
|
200 | (1) |
|
Missing in Action: Where Are the Data? |
|
|
201 | (2) |
|
In the Final Analysis, What Data Are Available? |
|
|
203 | (2) |
|
Outcomes With High Hopes Versus Reality That Modifies Them |
|
|
205 | (2) |
|
Data Collection and Evidence That Matters |
|
|
207 | (1) |
|
Summary of Chapter 6: The Research Consumer Evaluates Measurement Reliability and Validity |
|
|
208 | (2) |
|
|
208 | (2) |
|
|
210 | (1) |
|
|
211 | (2) |
|
Getting Closer: Grading the Literature and Evaluating the Strength of the Evidence |
|
|
213 | (52) |
|
Doing It Right: Reviewing the Literature From Locating Articles to Reporting Results |
|
|
215 | (1) |
|
The Spotlight Is on Quality |
|
|
215 | (4) |
|
RCTs: Consorting With the Best |
|
|
219 | (4) |
|
To Agree or Disagree: Defining Terms |
|
|
223 | (3) |
|
Observational or Nonrandomized Studies: Trend and Quality |
|
|
226 | (1) |
|
Scoring and Grading: Distinguishing Good From Poor Quality |
|
|
227 | (3) |
|
Quality, Quantity, and Consistency = Strength of Evidence |
|
|
230 | (4) |
|
The Evidence Makes the Grade: Time for Abstraction |
|
|
234 | (2) |
|
Bringing It Together: The Data Table |
|
|
236 | (1) |
|
Making the Grade: How to Identify Evidence That Matters |
|
|
236 | (6) |
|
|
241 | (1) |
|
|
241 | (1) |
|
Quality of the Literature |
|
|
241 | (1) |
|
|
242 | (1) |
|
Reliable and Valid Reviews |
|
|
242 | (4) |
|
Measuring Review Reliability: The Kappa Statistic |
|
|
244 | (2) |
|
Reviewing Other Reviews: Narrative Reviews and Systematic Reviews |
|
|
246 | (3) |
|
Systematic Reviews: Meta-Analysis |
|
|
249 | (5) |
|
|
249 | (1) |
|
How Many People and What Effect? |
|
|
250 | (1) |
|
Effect Size and Sample Size |
|
|
250 | (1) |
|
Effect Size, Odds and Risks |
|
|
251 | (3) |
|
A Checklist of Seven Questions to Guide in Evaluating the Quality of a Meta-Analysis |
|
|
254 | (6) |
|
Funding and Support: The Role of Sponsor |
|
|
260 | (1) |
|
Summary of Chapter 7: Getting Closer |
|
|
260 | (2) |
|
|
260 | (2) |
|
|
262 | (1) |
|
|
263 | (2) |
|
The Ethical Research Consumer Assesses Needs and Evaluates Improvement |
|
|
265 | (46) |
|
Identifying Needs, Preferences, and Values |
|
|
266 | (2) |
|
Techniques for Assessing Needs |
|
|
268 | (15) |
|
|
269 | (1) |
|
Public or Community Forum |
|
|
270 | (1) |
|
|
270 | (1) |
|
|
271 | (1) |
|
|
272 | (2) |
|
The RAND/UCLA Appropriateness Method |
|
|
274 | (1) |
|
|
275 | (8) |
|
Facing Uncertainty and Coming to Consensus |
|
|
283 | (2) |
|
Improvement Evaluations: How Are We Doing? Can We Do Better? |
|
|
285 | (3) |
|
Improvement and Effectiveness Evaluations: Two Purposes for One Discipline |
|
|
288 | (4) |
|
Research and Ethics: An Indomitable Connection |
|
|
292 | (1) |
|
Research and the Institutional Review Board |
|
|
292 | (5) |
|
|
293 | (1) |
|
Obtaining Informed Consent |
|
|
294 | (2) |
|
The Special Case of Evaluations That Are Exempt From IRB Approval |
|
|
296 | (1) |
|
Limits to Confidentiality |
|
|
296 | (1) |
|
|
297 | (4) |
|
Practicing Research and Ethics |
|
|
301 | (2) |
|
Summary of Chapter 8: The Ethical Research Consumer Assesses Needs and Evaluates Improvement |
|
|
303 | (4) |
|
|
303 | (4) |
|
|
307 | (4) |
Appendix: Answers to Exercises |
|
311 | (18) |
Glossary |
|
329 | (24) |
References |
|
353 | (8) |
Index |
|
361 | (12) |
About the Author |
|
373 | |