|
Part I Introduction to Intervention Effectiveness Research, Quality Improvement, and Program Evaluation |
|
|
|
1 Key Concepts, Definitions, and Frameworks |
|
|
3 | (14) |
|
|
3 | (1) |
|
1.2 Definitions and Descriptions of Intervention Effectiveness Research, Quality Improvement, and Program Evaluation: What They Have in Common and How They Differ |
|
|
4 | (1) |
|
1.2.1 What Is Quality Improvement? |
|
|
4 | (1) |
|
1.2.2 What Is Program Evaluation? |
|
|
5 | (1) |
|
1.3 How Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation Are Similar |
|
|
5 | (1) |
|
1.4 How Intervention Effectiveness Research, Quality Improvement, and Outcome Evaluation Are Different |
|
|
6 | (3) |
|
1.4.1 Translational Research |
|
|
6 | (1) |
|
1.4.2 Quality Improvement (QI) |
|
|
6 | (1) |
|
1.4.3 Six Sigma Quality Improvement |
|
|
7 | (1) |
|
1.4.4 Health Services Research |
|
|
7 | (1) |
|
1.4.5 Big Data in Health Care Research |
|
|
8 | (1) |
|
|
8 | (1) |
|
1.4.7 Implementation Research |
|
|
9 | (1) |
|
1.5 Definitions of Similar Sounding Terms and What This Book Does Not Attempt |
|
|
9 | (1) |
|
1.5.1 Comparative Effectiveness Research |
|
|
9 | (1) |
|
1.5.2 Implementation Science Research |
|
|
9 | (1) |
|
1.5.3 Dissemination Science |
|
|
10 | (1) |
|
1.6 Frameworks to Support Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation |
|
|
10 | (7) |
|
|
10 | (2) |
|
|
12 | (1) |
|
1.6.3 Theoretical Framework |
|
|
12 | (1) |
|
1.6.4 Conceptual Framework |
|
|
12 | (1) |
|
|
13 | (4) |
|
2 Problem-Intervention-Outcome Meta-Model (PIO MM): A Conceptual Meta Model for Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation |
|
|
17 | (12) |
|
2.1 Introduction to the Problem-Intervention-Outcome Meta-Model (PIO MM) |
|
|
17 | (2) |
|
2.2 PIO MM and the CDC Logic Model |
|
|
19 | (1) |
|
2.3 PIO MM and the IHI Quality Improvement Model |
|
|
20 | (2) |
|
|
22 | (3) |
|
2.5 Operationalizing the PIO MM |
|
|
25 | (1) |
|
2.6 PIO MM Relationship to Change Theory |
|
|
26 | (1) |
|
2.7 PIO MM Relationship to PICOT |
|
|
26 | (3) |
|
|
27 | (2) |
|
3 Problem-Intervention-Outcome Meta-Model Project Design |
|
|
29 | (12) |
|
3.1 Design for Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation |
|
|
29 | (3) |
|
3.1.1 Observational Design |
|
|
29 | (1) |
|
3.1.2 Retrospective Design |
|
|
30 | (1) |
|
|
31 | (1) |
|
3.2 Intervention and Measurement Timing |
|
|
32 | (1) |
|
3.3 PIO MM and Research Design |
|
|
32 | (1) |
|
3.4 Benefits and Challenges of the Single Group Before and After Design |
|
|
32 | (2) |
|
3.4.1 Threats to Internal Validity |
|
|
33 | (1) |
|
3.4.2 Enhancing Before and After Design Using Comparisons |
|
|
33 | (1) |
|
3.4.3 Considerations for Prospective Data Collection |
|
|
34 | (1) |
|
3.5 Comparisons Using PIO MM Variables |
|
|
34 | (3) |
|
|
34 | (1) |
|
|
34 | (1) |
|
|
35 | (1) |
|
|
36 | (1) |
|
3.5.5 Population (Individual Characteristics) |
|
|
36 | (1) |
|
|
36 | (1) |
|
|
37 | (1) |
|
3.6 Mixed Methods: Qualitative Evaluation |
|
|
37 | (4) |
|
|
38 | (3) |
|
4 Tools for Intervention Effectiveness Research, Quality Improvement Activities, and Program Evaluation |
|
|
41 | (12) |
|
|
41 | (1) |
|
4.2 Checklists for Obtaining New or Existing Data for Operationalizing the PIO MM |
|
|
42 | (2) |
|
4.3 Electronic Health Record Data |
|
|
44 | (1) |
|
4.4 Nursing-Specific Data |
|
|
44 | (1) |
|
|
45 | (3) |
|
4.5.1 Problem Classification Scheme |
|
|
46 | (1) |
|
4.5.2 Intervention Scheme |
|
|
46 | (1) |
|
4.5.3 Problem Rating Scale for Outcomes |
|
|
46 | (2) |
|
4.6 Analysis Software and Techniques |
|
|
48 | (1) |
|
|
48 | (1) |
|
4.8 Software for Descriptive and Inferential Statistical Methods and for Creating Graphs/Charts |
|
|
48 | (1) |
|
|
48 | (1) |
|
|
49 | (1) |
|
|
49 | (1) |
|
4.9 Big Data (Pattern Detection) Methods |
|
|
49 | (2) |
|
|
50 | (1) |
|
|
50 | (1) |
|
|
51 | (2) |
|
|
51 | (2) |
|
5 Descriptive Analysis and Interpretation |
|
|
53 | (10) |
|
|
53 | (1) |
|
|
53 | (2) |
|
|
54 | (1) |
|
|
54 | (1) |
|
|
55 | (1) |
|
|
55 | (1) |
|
|
55 | (2) |
|
5.3.1 Transforming and Recoding |
|
|
55 | (1) |
|
5.3.2 Identification and Labeling of Clusters Within a Sample |
|
|
56 | (1) |
|
5.4 Descriptive Statistics |
|
|
57 | (6) |
|
|
58 | (1) |
|
5.4.2 Cross Tabulation (Cross Tab) Matrix |
|
|
59 | (1) |
|
|
59 | (1) |
|
5.4.4 Measures of Central Tendency |
|
|
59 | (2) |
|
5.4.5 Measures of Distribution |
|
|
61 | (1) |
|
|
62 | (1) |
|
6 Inferential Analysis and Interpretation |
|
|
63 | (14) |
|
6.1 About Inferential Statistics |
|
|
63 | (3) |
|
6.2 Comparisons and Statistical Significance |
|
|
66 | (3) |
|
6.2.1 Comparisons of Sample Characteristics |
|
|
66 | (1) |
|
6.2.2 Outcomes as Measured by Before and After Comparison |
|
|
66 | (2) |
|
|
68 | (1) |
|
6.2.4 The P-Value in Large Dataset Research |
|
|
69 | (1) |
|
6.3 Clinical or Practical Significance |
|
|
69 | (1) |
|
6.3.1 Effect Size (Clinical or Practical Significance of Pchange= PTime2-- PTime1) |
|
|
69 | (1) |
|
6.3.2 Interpretation of Effect Size (Clinical or Practical Significance) |
|
|
70 | (1) |
|
|
70 | (3) |
|
|
70 | (2) |
|
|
72 | (1) |
|
6.4.3 Interpretation of Correlations |
|
|
72 | (1) |
|
6.4.4 Survival Analysis (PTime1, PTime2, ... PTimeX) |
|
|
72 | (1) |
|
6.4.5 Cross Tabs and Chi-Square (x2) |
|
|
73 | (1) |
|
|
73 | (4) |
|
|
75 | (2) |
|
7 Exploratory Data Analysis |
|
|
77 | (10) |
|
7.1 The Development of Exploratory Data Analysis |
|
|
77 | (1) |
|
7.2 Interpretation of Exploratory Data Analysis |
|
|
78 | (1) |
|
7.3 Visualization Techniques |
|
|
78 | (9) |
|
|
78 | (2) |
|
|
80 | (4) |
|
|
84 | (3) |
|
|
87 | (12) |
|
|
87 | (1) |
|
|
88 | (4) |
|
8.2.1 Where and How to Access an IRB |
|
|
88 | (1) |
|
8.2.2 When a Project May Be Exempt from IRB Review |
|
|
88 | (1) |
|
8.2.3 The Special Case of Quality Improvement |
|
|
89 | (1) |
|
8.2.4 Minimal Risk and IRB Review |
|
|
90 | (1) |
|
8.2.5 The Special Case of Program Evaluation |
|
|
91 | (1) |
|
|
92 | (1) |
|
8.3.1 What Is Informed Consent? |
|
|
92 | (1) |
|
8.3.2 Informed Consent Processes in the Context of Existing Data |
|
|
93 | (1) |
|
8.4 Data Privacy and Security |
|
|
93 | (6) |
|
|
95 | (4) |
|
Part II Practical Guide for Using the Problem-Intervention-Outcome Meta-Model |
|
|
|
9 Use the Worksheets and PIO MM Figure |
|
|
99 | (8) |
|
|
99 | (1) |
|
|
100 | (1) |
|
9.2.1 Examples of Projects |
|
|
100 | (1) |
|
|
101 | (6) |
|
|
101 | (4) |
|
9.3.2 Complete the PIO MM Diagram |
|
|
105 | (1) |
|
|
105 | (2) |
|
10 Know the Literature (Worksheet A) |
|
|
107 | (12) |
|
10.1 Preparing to Complete Worksheet A |
|
|
107 | (1) |
|
10.2 Step-by Step Instructions for Completing Worksheet A |
|
|
108 | (7) |
|
10.2.1 Population of Interest |
|
|
109 | (1) |
|
|
110 | (1) |
|
10.2.3 Measure(s) of Outcome |
|
|
110 | (1) |
|
10.2.4 Intervention(s) Used |
|
|
111 | (1) |
|
10.2.5 Measures of Intervention |
|
|
112 | (1) |
|
10.2.6 Measure of Intervention Fidelity |
|
|
112 | (1) |
|
10.2.7 Demographic Characteristics of a Sample |
|
|
113 | (1) |
|
10.2.8 Contextual Factors |
|
|
113 | (1) |
|
|
114 | (1) |
|
|
115 | (1) |
|
10.2.11 Complete Reference |
|
|
115 | (1) |
|
10.3 Sources of Information for the PIO MM Matrix |
|
|
115 | (4) |
|
|
118 | (1) |
|
11 Define the Problem (Worksheet B) |
|
|
119 | (12) |
|
11.1 Preparing to Complete Worksheet B |
|
|
119 | (1) |
|
11.2 Step-by Step Instructions for Completing Worksheet B |
|
|
120 | (11) |
|
|
120 | (1) |
|
11.2.2 Definition of the Problem |
|
|
121 | (1) |
|
11.2.3 Population of Interest |
|
|
121 | (1) |
|
|
121 | (1) |
|
11.2.5 Problem Measurement Instrument/Scale |
|
|
122 | (1) |
|
11.2.6 Anticipated Outcome and Rationale |
|
|
123 | (1) |
|
11.2.7 What is Not Known/Gap in Knowledge |
|
|
123 | (5) |
|
|
128 | (3) |
|
12 Describe the Intervention (Worksheet C) |
|
|
131 | (12) |
|
12.1 Preparing to Complete Worksheet C |
|
|
131 | (1) |
|
12.2 Step-by Step Instructions for Completing Worksheet C |
|
|
132 | (11) |
|
12.2.1 Describe the Intervention |
|
|
132 | (1) |
|
12.2.2 Expected Effectiveness |
|
|
132 | (1) |
|
12.2.3 Theoretical Framework |
|
|
133 | (1) |
|
12.2.4 Intervention Content and Essential Core Components |
|
|
133 | (1) |
|
12.2.5 Describe Intervention Measurement: Amount, Type, Fidelity, Quality |
|
|
134 | (1) |
|
12.2.6 Describe Interventionist Characteristics: Qualifications, Training, Demographics |
|
|
135 | (6) |
|
|
141 | (2) |
|
13 Define the Outcome (Worksheet D) |
|
|
143 | (12) |
|
13.1 Preparing to Complete Worksheet D |
|
|
143 | (1) |
|
13.2 Step-by-Step Instructions for Completing Worksheet D |
|
|
144 | (11) |
|
|
152 | (3) |
|
14 Plan the Analysis (Worksheet E) |
|
|
155 | (12) |
|
14.1 Preparing to Complete Worksheet E |
|
|
155 | (5) |
|
14.1.1 Step 1. Review Project Statements |
|
|
156 | (2) |
|
14.1.2 Step 2. Select Statements That Are Most Applicable to the Project and Discipline |
|
|
158 | (1) |
|
14.1.3 Step 3. Review Design Options |
|
|
158 | (1) |
|
14.1.4 Step 4. State the Design |
|
|
158 | (1) |
|
14.1.5 Step 5. Review Variables |
|
|
159 | (1) |
|
14.1.6 Step 6. Plan for Creating New Variables |
|
|
159 | (1) |
|
14.2 Step-by-Step Instructions for Completing Worksheet E |
|
|
160 | (7) |
|
14.2.1 Exploratory Data Analysis |
|
|
160 | (1) |
|
|
160 | (1) |
|
|
161 | (1) |
|
|
162 | (1) |
|
14.2.5 Relationships Among Variables |
|
|
163 | (1) |
|
|
164 | (3) |
|
15 Interpret the Results (Worksheet F) |
|
|
167 | (12) |
|
15.1 Preparing to Complete Worksheet F |
|
|
167 | (1) |
|
15.2 Results Statements and Presentation |
|
|
168 | (7) |
|
15.2.1 Presenting the Results |
|
|
168 | (1) |
|
15.2.2 Description of Sample Characteristics |
|
|
168 | (2) |
|
15.2.3 Description of Interventions |
|
|
170 | (1) |
|
15.2.4 Description of Outcomes |
|
|
171 | (2) |
|
15.2.5 Description of Benchmark Attainment |
|
|
173 | (1) |
|
15.2.6 Correlations Between Interventions and Outcomes |
|
|
174 | (1) |
|
15.3 Results Interpretation |
|
|
175 | (4) |
|
15.3.1 Theoretical Framework-Related Interpretation |
|
|
176 | (1) |
|
15.3.2 Temporality-Related Interpretation |
|
|
176 | (1) |
|
15.3.3 Give Alternative Explanations for the Findings |
|
|
177 | (1) |
|
|
178 | (1) |
|
16 Disseminate the Findings |
|
|
179 | (26) |
|
16.1 Why Dissemination Matters |
|
|
179 | (1) |
|
16.2 Getting the Most Benefit from This Chapter |
|
|
180 | (1) |
|
16.3 Iterative Interpretation and Explication of the Overall Story |
|
|
180 | (4) |
|
16.4 Drafting the Abstract: Summarize the Story in Brief |
|
|
184 | (1) |
|
16.5 Develop and Display Results |
|
|
185 | (2) |
|
16.6 Adding Meaningful Interpretation to the Results |
|
|
187 | (6) |
|
|
193 | (1) |
|
|
194 | (1) |
|
16.9 The Purpose Statement |
|
|
195 | (1) |
|
16.10 Background to Set the Stage for the Purpose |
|
|
195 | (2) |
|
16.11 The Gap in Knowledge |
|
|
197 | (1) |
|
16.12 Title, Abstract, and Conclusion |
|
|
198 | (1) |
|
16.13 Rewrite the Abstract |
|
|
199 | (1) |
|
16.14 Write the Conclusions Section |
|
|
199 | (2) |
|
|
201 | (1) |
|
16.16 Styles and Author Guidelines |
|
|
201 | (4) |
|
|
202 | (3) |
|
17 Synthesis, Next Steps, and Epilogue |
|
|
205 | |
|
|
205 | (2) |
|
17.2 Questions to Inspire Next Steps |
|
|
207 | (1) |
|
17.3 Building Evidence on Evidence |
|
|
208 | |
|
|
209 | |