Evaluation Fundamentals: Insights into the Outcomes, Effectiveness, and Quality of Health Programs 2nd Revised edition [Pehme köide]

Preface		xi
Program Evaluation: A Prelude		3	(38)
A Reader's Guide to Chapter 1		3	(1)
What Is Program Evaluation?		4	(2)
How to Evaluate: The Methods		4	(1)
The Program or Intervention		4	(1)
Program Merits or Standards of Effectiveness		4	(1)
Program Objectives and Activities		5	(1)
Program Outcomes		5	(1)
Program Impact		6	(1)
Program Expenses and Costs		6	(1)
Users and Doers of Evaluations		6	(1)
The Composition of Program Evaluation		7	(3)
Posing Evaluation Questions		7	(1)
Setting Standards of Effectiveness		8	(1)
Designing the Study		8	(1)
Selecting the Participants		9	(1)
Collecting Data		9	(1)
Managing Data		10	(1)
Analyzing Data		10	(1)
Reporting the Results		10	(1)
Baseline Data, Interim Data (Formative Evaluation), and Process Evaluations		10	(4)
Baseline Data		11	(1)
Interim Data and Formative Evaluation		11	(2)
Process or Implementation Evaluation		13	(1)
Qualitative Evaluation		14	(1)
Participatory and Community-Based Evaluations		15	(2)
Evaluation Frameworks and Models		17	(3)
Ethical Evaluations		20	(7)
Evaluations That Need Institutional Review Board Approval		20	(1)
Evaluations That Are Exempt From IRB Approval		21	(1)
What the IRB Will Review		22	(1)
Informed Consent		23	(1)
Purpose		23	(1)
Contents of an Informed Consent Form to Participate in an Evaluation		24	(3)
Limits to Confidentiality		27	(1)
Confidentiality of Audio-/Videotapes and Photographs		27	(1)
Research Misconduct		27	(3)
A Note About the Health Insurance Portability and Accountability Act		28	(1)
Protected Health Information		29	(1)
Summary and Transition to the Next Chapter on Evaluation Questions and Standards of Effectiveness		30	(1)
Exercises		31	(5)
Suggested Readings		36	(5)
Evaluation Questions and Standards of Effectiveness		41	(30)
A Reader's Guide to Chapter 2		41	(1)
Evaluation Questions		42	(6)
Goals and Objectives		42	(1)
Participants and Effectiveness		43	(1)
Program Activities, Organization, and Effectiveness		44	(1)
Economics and Costs		45	(2)
Program Environment		47	(1)
Setting Standards of Effectiveness: What They Are and How to State Them		48	(3)
How to Set Standards		51	(10)
Setting Standards Using Comparisons With Other Programs		52	(2)
Setting Standards Using Experts		54	(3)
Setting Standards Using Community Data Sets		57	(1)
Setting Standards Using the Literature		58	(2)
Evaluation Standards and Economic Evaluations		60	(1)
Evaluation Questions and Standards: Establishing a Healthy Relationship		61	(1)
When to Set Standards		62	(1)
The QSV Report: Questions, Standards, Variables		63	(2)
Summary and Transition to the Next Chapter on Evaluation Research Design		65	(1)
Exercises		66	(1)
Suggested Readings		67	(4)
Designing Program Evaluations		71	(28)
A Reader's Guide to Chapter 3		71	(1)
Evaluation Design: Creating the Structure		72	(3)
Selecting an Evaluation Design: Dealing With Six Questions		75	(7)
What Are the Evaluation Questions and Standards?		75	(1)
What Are the Independent Variables?		75	(1)
Relationships Among Evaluation Questions, Standards, Independent Variables, and Outcomes		75	(1)
Evaluation Questions, Standards, Independent Variables, Outcomes, and Design		76	(1)
What Are the Inclusion and Exclusion Criteria?		77	(2)
Will a Control Group Be Included?		79	(1)
When Will Measures Take Place?		80	(1)
How Often Will Measures Take Place?		80	(2)
Designs for Program Evaluation		82	(1)
A Classification of Program Evaluation Designs		83	(10)
Experimental Evaluations		84	(1)
Evaluations With Concurrent Controls and Random Assignment		84	(1)
Evaluations With Concurrent Controls But No Randomization		84	(1)
Evaluations With Self-Controls		85	(1)
Evaluations With Historical Controls		86	(1)
Descriptive or Observational Evaluation Designs		86	(1)
Cross-Sectional Designs		86	(1)
Cohort Designs		87	(1)
Either/Or: Factorial Designs		88	(1)
Descriptive Factorial Designs		88	(2)
Experimental Factorial Designs		90	(1)
Combinations: The Solomon Four-Group Design		91	(2)
Internal and External Validity		93	(1)
The Evaluation Design Report: Questions, Standards, and Independent Variables		93	(2)
Summary and Transition to the Next Chapter on Sampling		95	(1)
Exercise		96	(1)
Suggested Readings		96	(3)
Sampling		99	(18)
A Reader's Guide to Chapter 4		99	(1)
What Is a Sample?		100	(1)
Inclusion and Exclusion Criteria or Eligibility		101	(1)
Sampling Methods		102	(6)
Simple Random Sampling		102	(2)
Random Selection and Random Assignment		104	(1)
Systematic Sampling		105	(1)
Stratified Sampling		105	(2)
Cluster Sampling		107	(1)
Nonprobability or Convenience Sampling		107	(1)
The Sampling Unit		108	(1)
Sample Size		109	(2)
Power Analysis and Alpha and Beta Errors		109	(2)
Why Sample?		111	(1)
The Sampling Report		111	(2)
Summary and Transition to the Next Chapter on Collecting Information		113	(1)
Exercise		114	(1)
Suggested Readings		115	(2)
Collecting Information: The Right Data Sources		117	(28)
A Reader's Guide to Chapter 5		117	(1)
Information Sources: What's the Problem?		118	(1)
Choosing the Best Sources of Data		119	(2)
Sources of Data in Program Evaluation and Their Advantages and Limitations		121	(12)
Self-Administered Surveys		121	(2)
Tests of Achievement		123	(1)
Medical Record Reviews		124	(1)
Observations		125	(1)
Interviews		126	(1)
Computer-Assisted Interviews		127	(2)
Physical Examinations		129	(1)
Large Data Sets		129	(1)
Performance Tests: The Standardized Patient		130	(1)
Clinical Scenarios		131	(1)
The Literature		132	(1)
Guidelines for Reviewing the Literature		133	(7)
Assemble the Literature		134	(1)
Identify Inclusion and Exclusion Criteria		134	(1)
Select the Relevant Literature		135	(1)
Identify the ``Best'' Literature		136	(1)
Abstract the Information		137	(1)
Consider the Unpublished Literature		137	(3)
Summary and Transition to the Next Chapter on Evaluation Measures		140	(1)
Exercises		141	(1)
Suggested Readings		142	(3)
Evaluation Measures		145	(20)
A Reader's Guide to Chapter 6		145	(1)
Reliability and Validity		146	(3)
Reliability		146	(1)
Validity		147	(2)
A Note on Language: Data Collection Terms		149	(1)
Checklist for Creating a New Measure		150	(3)
Checklist for Selecting an Already Existing Measure		153	(2)
The Measurement Chart: Logical Connections		155	(3)
Summary and Transition to the Next Chapter on Managing Evaluation Data		158	(2)
Exercises		160	(2)
Suggested Readings		162	(3)
Managing Evaluation Data		165	(22)
A Reader's Guide to Chapter 7		165	(1)
Data Management: The Road to Data Analysis		166	(1)
Drafting an Analysis Plan		167	(1)
Creating a Codebook		168	(4)
Establishing Reliable Coding		170	(1)
Measuring Agreement: The Kappa		171	(1)
Searching for Missing Data		172	(3)
Entering the Data		175	(1)
Cleaning the Data		176	(4)
Information Collection Measures Not Completed		177	(2)
Information Collection Measures Only Partially Completed		179	(1)
Outliers		179	(1)
Data in Need of Recoding		179	(1)
Creating the Final Data Set for Analysis		180	(1)
Storing and Archiving the Data Set		181	(1)
Summary and Transition to the Next Chapter on Data Analysis		181	(2)
Exercises		183	(1)
Suggested Readings		184	(3)
Analyzing Evaluation Data		187	(32)
A Reader's Guide to Chapter 8		187	(1)
A Suitable Analysis: Starting With the Evaluation Questions		188	(1)
Measurement Scales and Their Data		188	(3)
Nominal Scales		189	(1)
Ordinal Scales		189	(2)
Numerical Scales		191	(1)
Selecting a Method of Analysis		191	(2)
Hypothesis Testing and p Values: Statistical Significance		193	(5)
Clinical or Practical Significance: Using Confidence Intervals		198	(3)
Establishing Clinical or Practical Significance		199	(2)
Proceed With Caution: Screening and Transforming Data		201	(1)
Risk and Odds		202	(3)
Odds Ratios and Relative Risk		203	(2)
Qualitative Evaluation Data: Content Analysis		205	(6)
Assembling the Data		206	(2)
Learning the Contents of the Data		208	(1)
Creating a Codebook		208	(1)
Entering and Cleaning the Data		209	(1)
Doing the Analysis		210	(1)
Meta-Analysis: A Superanalysis of Evaluation Data		211	(2)
Summary and Transition to the Next Chapter on Evaluation Reports		213	(1)
Exercises		214	(2)
Suggested Readings		216	(3)
Evaluation Reports		219	(28)
A Reader's Guide to Chapter 9		219	(1)
The Written Evaluation Report		220	(12)
Elements of the Report		222	(1)
Introduction		222	(1)
Methods		223	(1)
Results		224	(3)
Conclusions or Discussion		227	(2)
Recommendations		229	(1)
The Abstract		229	(2)
The Executive Summary		231	(1)
Reviewing Your Report		232	(6)
Oral Presentations		238	(6)
Recommendations for the Preparation of Computer ``Slide'' Presentations		238	(6)
Exercises		244	(1)
Suggested Readings		245	(2)
Answers to Exercises		247	(10)
Index		257	(8)
About the Author		265

Arlene Fink (PhD) is Professor of Medicine and Public Health at the University of California, Los Angeles, and president of the Langley Research Institute. Her main interests include evaluation and survey research and the conduct of research literature reviews as well as the evaluation of their quality. Dr. Fink has conducted scores of evaluation studies in public health, medicine, and education. She is on the faculty of UCLAs Robert Wood Johnson Clinical Scholars Program and is a scientific and evaluation advisor to UCLAs Gambling Studies and IMPACT (Improving Access, Counseling & Treatment for Californians with Prostate Cancer) programs. She consults nationally and internationally for agencies such as Linstitut de Promotion del la Prévention Secondaire en Addictologie (IPPSA) in Paris, France, and Peninsula Health in Victoria, Australia. Professor Fink has taught and lectured extensively all over the world and is the author of more than 130 peer-reviewed articles and 15 textbooks.