Muutke küpsiste eelistusi

Mismeasuring Impact: How Randomized Controlled Trials Threaten the Nonprofit Sector New edition [Kõva köide]

  • Formaat: Hardback, 206 pages, kõrgus x laius: 229x152 mm, 1 table, 10 figures
  • Sari: Stanford Social Innovation Review Books
  • Ilmumisaeg: 15-Jul-2025
  • Kirjastus: Stanford Business Books,US
  • ISBN-10: 1503634655
  • ISBN-13: 9781503634657
  • Formaat: Hardback, 206 pages, kõrgus x laius: 229x152 mm, 1 table, 10 figures
  • Sari: Stanford Social Innovation Review Books
  • Ilmumisaeg: 15-Jul-2025
  • Kirjastus: Stanford Business Books,US
  • ISBN-10: 1503634655
  • ISBN-13: 9781503634657
"Demonstrating the effectiveness of nonprofit social programs is a challenge. This has led to a rapid rise in nonprofit organizations' use of randomized controlled trials (RCTs), for evaluation. As a result, most nonprofit sector professionals today can tell you why nonprofits should do an RCT. This book tells you why they probably shouldn't. Mismeasuring Impact explores why RCTs are being embraced as the "gold standard" for nonprofit evaluation, despite the high cost and time investment required, and the problems with using RCTs in a nonprofit context. The book describes what happens inside nonprofits when they take part in RCTs, the unintended equity issues that arise, and why nonprofits decide to participate in RCTs despite the problems. University ofChicago professors Marwell and Mosley's research is based on extensive interviews with key players: nonprofit managers, professional program evaluators, and program officers in philanthropic foundations. The book explains why most RCTs conducted in nonprofits fail to meet required standards for rigor, undercutting their accuracy. Utlimately, RCTs are used to create nonprofit legitimacy, not to foster nonprofit improvement. RCTs also privilege program and organizational standardization over the key strengths of nonprofit organizations: innovation and responsiveness to community needs. Nonprofits and funders need forms of evaluation that lift up these strengths, but RCTs fall short. Mismeasuring Impact offers alternatives that build strong organizations, not just standardized programs. The book concludes by suggesting evaluation approaches that funders and nonprofits of all sizes can support"--

The hidden dangers of randomized controlled trials

The need to demonstrate the effectiveness of nonprofit social programs has led to a rapid rise in the use of randomized controlled trials (RCTs), for evaluation. As a result, most nonprofit sector professionals can tell you why nonprofits should do an RCT. This book tells you why they probably shouldn't, and what to do instead.

Mismeasuring Impact explores why RCTs are being embraced as the "gold standard" for nonprofit evaluation, despite the high cost and time investment required and the serious problems with using RCTs in a nonprofit context. (Most RCTs conducted in nonprofits fail to meet required standards for rigor, undercutting their accuracy). The book describes what happens inside nonprofits when they take part in RCTs, the unintended equity issues that arise, and why nonprofits feel pressured participate in RCTs despite the problems.

University of Chicago professors Marwell and Mosley's research is based on extensive interviews with key players: nonprofit managers, professional program evaluators, and program officers in philanthropic foundations. The book argues that, ultimately, RCTs are used to poorly ground nonprofit legitimacy, not to foster nonprofit improvement. RCTs also privilege program and organizational standardization over the key strengths of nonprofit organizations: flexible innovation and responsiveness to community needs.

Nonprofits and funders need forms of evaluation that lift up these strengths. Mismeasuring Impact offers alternative approaches that build strong organizations, not just cookie-cutter programs, and which funders and nonprofits of all sizes can support.

Arvustused

"Marwell and Mosley challenge the prevailing myth that RCTs are the 'gold standard' for evaluation, revealing RCTs as a movement rather than an infallible method. A must-read for nonprofit managers, funders, and anyone seeking a deeper, more nuanced approach to measuring impact." Johanna Mair, Professor of Organization, Strategy and Leadership at the Hertie School "Finally, a book that puts RCTs in their proper place: an evaluation method that's useful only under limited conditions. Marwell and Mosley show how the uncritical use of RCTs can harm nonprofitsby undermining relationships with communities, sapping staff morale, and putting organizations at risk. The gold standard is now, more fittingly, just a hammer in the evaluation toolbox." Alnoor Ebrahim, Professor of Management at Tufts University "In an exemplary work of humane and rigorous scholarship, Marwell and Mosley stand athwart the rise of the RCT as the dominant form of evidence for nonprofit impact and yell: 'Stop!' Even if you are a devotee of the RCTespecially if you onethis is an essential book to learn from and wrestle with." Benjamin Soskis, Center on Nonprofits and Philanthropy, The Urban Institute "A courageous, timely, and well-written warning against fashionable methodological assumptions in nonprofit evaluation. Marwell and Mosley demonstrate the not-so-glittering consequences of the so-called gold standard in evaluation." Peter Dahler-Larsen, University of Copenhagen. Author of The Evaluation Society. "Though I did not agree with everything here, this book should be required reading for thoughtful evaluators who want to 'do no harm' and desire to adopt the right tool in judicious partnership with funders, program operators, and community members." Matthew Stagner, Former president of the Association for Public Policy Analysis and Management "Rather than succumbing to pressures to embrace the Randomized Controlled Trial (RCT) 'gold standard,' this book sets the record straight with a clear description of how we got here and guidance for more effective evaluation. Essential reading for social sector leaders who want to 'do good better,' and social scientists who want to help them." Jodi Sandfort, Dean at the Evans School of Public Policy and Governance, at the University of Washington "Accurately measuring the impact of nonprofit organizations remains the 'Holy Grail' for many. Yet, there are many reasons why, all these years on, we are still short of consensus on the best way to do that work. Marwell & Mosley add to our shared understanding of how one such method Randomized Controlled Trials may not be the 'gold standard' once advertised. While measuring impact is important, we must not lose sight of the inputs needed to ensure nonprofits are healthy and equipped to deliver the impact we all desire." Jeffrey Moore, Chief Strategy Officer at Independent Sector "Marwell and Mosley show that RCTs cannot capture the complexity of the real world and undermine important nonprofit contributions to communities and society as a result. Mismeasuring Impact is necessary reading for scholars doing policy-relevant research and for policymakers who want to base their decisions on evidence." Mary Pattillo, author of Black on the Block: The Politics of Race and Class in the City, Harold Washington Professor of Sociology at Northwestern University "RCT's have quickly emerged as the preferred evaluation tool for many nonprofit programs. Yet Marwell and Mosley, in their timely, outstanding book, persuasively call attention to the programmatic and organizational drawbacks of RCTs that can undermine nonprofit effectiveness, responsiveness, and capacity for innovation." Steven Rathgeb Smith, McCourt School of Public Policy, Georgetown University, past president of the Association for Research on Nonprofit and Voluntary Action (ARNOVA) and the International Society for Third Sector Research (ISTR) "As experts in sociology, social work, and the nonprofit sector, the authors offer an illuminating and sobering critique of randomized controlled trials (RCTs) for charitable organizations. This book will appeal to academics, policymakers, and practitioners alike." David Suárez, Colleen Willoughby Endowed Faculty Fellow in Philanthropy & Civil Society, Evans School of Public Policy and Governance, at the University of Washington

Acknowledgments
1. FIVE PROBLEMS WITH RCTs, AND WHY THEY THREATEN THE NONPROFIT SECTOR
The Case for RCTs
Are RCTs Really the "Gold Standard" for Nonprofits?
Five Problems with Using RCTs in Nonprofit Organizations
Goals of This Book
Brief Answers to Some Common Objections to Our Argument
2. THE SIREN SONG OF RCTs: BUILDING THE GOLD STANDARD MOVEMENT
How Did We Get Here? Understanding the Growth of RCTs for Social Programs
The Evidence Battle
The Funding Battle
Spreading RCTs to U.S. Nonprofits: The Social Innovation Fund
Changing the Conversation
3. NULL EFFECTS AND DUMB EFFECTS: DO RCTs REALLY TELL US "WHAT WORKS?"
The Ubiquity of Null Effects
Dumb Effects: When Positive Effects Are Not Very Meaningful
Moving Beyond RCTs
4. "THIS WAS HARDER THAN WE THOUGHT": IMPLEMENTING RCTs IN NONPROFITS
Twice the People, Twice the Work: RCT Recruiting Challenges Staff Capacity
and Organizational Mission
"It Sounded a Lot Easier Than It Actually Was"
"It Sounded a Lot Easier Than It Actually Was"
Delays and (Financial) Drains
RCTs Don't Just Evaluate, They Can Change Both Programs and Organizations
Putting Organizational Well-Being First
5. LOOKING FOR LEGITIMACY: WHY WE KEEP DOING RCTs
RCTs Can Build Organizational Legitimacy
Does an RCT Always Enhance Nonprofit Legitimacy?
Evaluators See Limits on RCTs
The View from Philanthropy
RCTs May Increase Legitimacy, but Not Without Consequences
6. WHAT KIND OF WORK SHOULD WE VALUE? HOW RCTs UNDERMINE THE EQUITY GOALS
OF NONPROFITS AND THE COMMUNITIES THEY SERVE
Is Pursuing Effectiveness the Same as Pursuing Equity?
Do RCTs Privilege Evidence-Building Over People?
The Rich Get Richer
Practicing Equity: A Key Goal for Philanthropy
Evaluation, Effectiveness, and Equity
7. A BAD FIT :HOW RCTs HINDER NONPROFIT RESPONSIVENESS AND INNOVATION
Long Time Horizon
The "Hold Still" Challenge: RCTs Limit Ongoing Program Responsiveness and
Innovation
"You Get One Shot"
A Black Box
The Need for Alternatives to the RCT . . . and More Evaluation Capacity
Overall
Evaluation Should Better Support Responsiveness and Innovation
8. HOW MIGHT WE THINK ABOUT EVALUATION DIFFERENTLY? THREE PRINCIPLES FOR
NONPROFIT IMPROVEMENT
Principle 1: Tailor Evaluation to Specific Organizational Strategies and
Community Needs
Principle 2: Centering Participant Perspectives Is Key to Nonprofit
Performance
Principle 3: Focus on Improvement Through Iteratively Addressing Problems
Instead of Standardizing Solutions
An Improvement Orientation
9. MOVING BEYOND RCTs: LESSONS FOR NONPROFIT LEADERS, DONORS, EVALUATORS,
AND POLICYMAKERS
Evidence Matters
Five Problems with RCTs
When Is the Right Time to Do an RCT?
Three Principles for Using Evaluation to Foster Nonprofit Improvement
Recommendations for Building Evidence Outside the Evidence Hierarchy
The Way Forward
Appendix: Research Methodology
Notes
Bibliography
Index
Nicole P. Marwell and Jennifer E. Mosley are Professors at the Crown Family School of Social Work, Policy, and Practice at the University of Chicago. Their research on nonprofit organizations has been published widely in leading journals in the fields of nonprofit studies, sociology, public administration, and social work.