HUMAN DEVELOPMENT PERSPECTIVES Barbara Bruns, Deon Filmer, and Harry Anthony Patrinos Making Schools Work New Evidence on Accountability Reforms Making Schools Work Making Schools Work New Evidence on Accountability Reforms Barbara Bruns, Deon Filmer, and Harry Anthony Patrinos © 2011 The International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org All rights reserved 1 2 3 4 :: 14 13 12 11 This volume is a product of the staff of the International Bank for Reconstruction and Development / The World Bank. The fi ndings, interpretations, and conclusions expressed in this volume do not necessarily refl ect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgement on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development / The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with com- plete information to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Offi ce of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. ISBN: 978-0-8213-8679-8 eISBN: 978-0-8213-8680-4 DOI: 10.1596/978-0-8213-8679-8 Library of Congress Cataloging-in-Publication Data Bruns, Barbara. Making schools work : new evidence on accountability reforms / Barbara Bruns, Deon Filmer, Harry Anthony Patrinos. p. cm. — (Human development perspectives) Includes bibliographical references. ISBN 978-0-8213-8679-8 (alk. paper) — ISBN 978-0-8213-8680-4 1. Educational tests and measurements—United States. 2. Educational accountability— United States. 3. Public schools—United States—Examinations. 4. School improvement pro- grams. 5. Educational leadership. I. Filmer, Deon. II. Patrinos, Harry Anthony. III. Title. LB3051.B78 2011 371.2'07—dc22 2010053396 Cover photos: Barbara Bruns/World Bank (sleeping teacher); Erica Amorim/World Bank (Brazilian teacher with students) Cover design: Naylor Design v Foreword ix Acknowledgments xi About the Authors xiii Abbreviations xv Chapter 1: Motivation and Framework 1 Service Delivery Failure in the Developing World 3 Three Core Strategies for More Accountable Education Systems 12 Accountability and Evidence 20 Notes 25 References 25 Chapter 2: Information for Accountability 29 How Information Can Increase Accountability—and Outcomes 33 Information for Accountability in High-Income Countries 38 Information for Accountability in Middle- and Low-Income Countries 41 Evaluating the Impact of Information-for- Accountability Interventions 49 What Have We Learned? 62 Contents vi|Contents Conclusion: Beyond Proof of Concept 73 Notes 78 References 80 Chapter 3: School-Based Management 87 Decentralization in School-Based Management 88 Toward a Theory of School-Based Management 90 Assessing the Evidence 102 Conclusions 122 Notes 134 References 135 Chapter 4: Making Teachers Accountable 141 Teacher Accountability Reforms: Why? 141 Recent Global Experience with Teacher Accountability Reforms 143 Contract Tenure Reforms 146 Pay-for-Performance Reforms 157 Designing Teacher Accountability Reforms 181 Summary and Conclusions 196 Annex: Rating the Design Features of Pay-for- Performance Programs 200 Notes 203 References 205 Chapter 5: Making Schools Work through Accountability Reforms 211 Information-for-Accountability Strategies 211 School-Based Management Strategies 218 Teacher Contracting and Pay-for-Performance Strategies 223 Linking Accountability Reforms 236 External Validity: From Evaluated Programs to National Reforms 239 The Political Economy of Service Delivery Reform 245 Future Directions 247 Note 248 References 248 Contents|vii Boxes 2.1 Using Aggregated Data for Accountability 31 2.2 Using Information for Management 37 2.3 Citizen Report Cards 43 2.4 Cost-Effectiveness of Information Interventions 64 3.1 Eight Years to See Results 101 3.2 Ongoing SBM Experiments in Africa 121 4.1 Targets that Avoid Perverse Incentives: Brazil’s Index of Basic Education Development 168 5.1 New Evidence on Information for Accountability 213 5.2 New Evidence on School-Based Management 219 5.3 New Evidence on Contract Teachers 224 5.4 New Evidence on Pay for Performance 228 Figures 1.1 Comparative PISA Math Profi ciency, 2009 4 1.2 Correlation of Education Spending to Student Performance 6 1.3 Shares of Public Education Spending Benefi ting the Richest and Poorest Population Quintiles, Selected Countries 8 1.4 Teacher Classroom Presence and Time Spent Teaching, Selected Countries 10 1.5 The Accountability Framework 11 1.6 Teacher Performance Incentives 18 2.1 The Role of Information in the Accountability Framework 33 2.2 Report Cards Given to Parents in Pakistan 67 2.3 Report Card in Paraná State, Brazil, 1999–2000 68 3.1 The Accountability Framework in School-Based Management 91 3.2 The Autonomy-Participation Nexus, Selected SBM Programs 94 3.3 From School-Based Management to Measurable Results 98 4.1 Teacher Performance Incentives 144 4.2 Comparison of Bonus-Pay Programs by Impact Size and Predicted Incentive Strength 194 4A.1 Bonus Predictability Ratings 202 4A.2 Bonus Size Ratings 203 5.1 SBM Results: A Meta-Analysis of U.S. Models 218 5.2 Complementarities in Accountability Reform 239 viii|Contents Tables 1.1 Percentage of School Grants Reaching Schools in Selected Countries 9 2.1 Impact Evaluation Studies of Information-for- Accountability Interventions 63 3.1 School-Based Management Reforms in Selected Countries 95 3.2 Intermediate Outcomes from SBM Reforms 97 3.3 Inside the Black Box: How to Measure the Impact of SBM Programs 100 3.4 Evaluations and Impacts: SBM Evidence from Recent Rigorous Studies 123 4.1 Summary of Evaluated Contract Tenure Reforms 148 4.2 Summary of Evaluated Pay-for-Performance (Bonus Pay) Reforms 160 4.3 Classroom Dynamics in 220 Pernambuco Schools, November 2009 173 4.4 Incentive Program Design Features and Possible Effects 187 4.5 Pay-for-Performance Programs by Core Design Features and Effect Size 191 [...]... decentralization of school-level decision making autonomy—to school-level agents • Teacher incentives: policies that link pay or tenure directly to performance Information for Accountability The notion that increased information in education can improve accountability and outcomes is not new In the 1990s, the education sector in the United States experienced a large-scale increase in test-based accountability. .. Education Reform Act, in 1988, empowered school communities by giving public secondary schools the option of leaving local-education-authority control and becoming autonomous, grant-maintained (GM) schools GM schools were funded by a new agency but were owned and managed by each school’s governing body: a new 1 0- to 15-member entity composed of the head teacher and teacher and parent representatives Control... Progress in International Reading Literacy Study Programme for International Student Assessment Partnership for Educational Revitalization in the Americas parent-teacher association ramdomized control trial [experimental method] regression discontinuity design [experimental method] school-based management standard deviation school development and monitoring committee [India] National System for Measuring... International Reading Literacy Study [PIRLS]) from more than 50 countries over a 40-year period, Hanushek and Woessmann (2007, 2010) have demonstrated a tight correlation between average student learning levels and long-term economic growth The relationship holds across high-income countries, across developing countries, across regions, and across countries Motivation and Framework | 5 within regions:... participation among the poor (such as conditional cash or food transfers) and interventions that aim to improve education service provision (such as policies to improve the quality of teachers in remote areas) He received his Ph.D in economics from Brown University Harry Anthony Patrinos is lead education economist in the Education Department of the World Bank He specializes in all areas of education, especially... Escolar) [Mexico] School Operational Assistance Program (Bantuan Operasional Sekolah) [Indonesia] Civic Engagement for Education Reform in Central America difference-in-differences [econometric method] Education with Community Participation (Educación con Participación de la Comunidad) Early Grade Reading Assessment [Liberia] Education Management Information System Education Quality Improvement Project... cognitive skills are consistently and highly correlated with long-term rates of per capita income growth While the quantity of education (average years of schooling of the labor force) is statistically significantly related to long-term economic growth in analyses that neglect education quality, the association between years of schooling and economic growth falls to close to zero once education quality (measured... core conundrum of education economics The World Development Report 2004 broke new ground on this question by looking broadly at the ways in which public spending in developing countries failed to result in quality services for clients, particularly the 6 | Making Schools Work Figure 1.2 Correlation of Education Spending to Student Performance a Primary math test scores vs global public education spendinga... figure includes only government-funded schools, and the unit cost includes teachers and book-related expenses Motivation and Framework | 7 poorest clients (World Bank 2003) It documented key issues in the “service delivery chain,” including inequitable allocation to low-income groups, the “leakage” of funding en route from central ministries to front-line providers, and the failure of front-line providers... in OECD, middle-income, and low-income countries over the past two decades, this book focuses on three widely used strategies that each have a clear rationale for how reforms might translate into improved learning outcomes: • Information for accountability: generation and dissemination of information about schooling rights and responsibilities, inputs, outputs, and outcomes • School-based management: . Deon Filmer, and Harry Anthony Patrinos Making Schools Work New Evidence on Accountability Reforms Making Schools Work Making Schools Work New Evidence. Washington, DC 20433, USA; fax: 20 2-5 2 2-2 422; e-mail: pubrights@worldbank.org. ISBN: 97 8-0 -8 21 3-8 67 9-8 eISBN: 97 8-0 -8 21 3-8 68 0-4 DOI: 10.1596/97 8-0 -8 21 3-8 67 9-8 Library