A framework for mandatory impact evaluation to ensure well informed public policy decisions

Trillions of dollars are invested yearly in programmes to improve health, social welfare, education, and justice (which we will refer to generally as public programmes). Yet we know little about the eff ects of most of these attempts to improve peoples’ lives, and what we do know is often not used to inform decisions. We propose that governments and non-governmental organisations (NGOs) address this failure responsibly by mandating more systematic and transparent use of research evidence to assess the likely eff ects of public programmes before they are launched, and the better use of well designed impact evaluations after they are launched. Resources for public programmes will always be scarce. In low-income and middle-income countries, where there are often particularly severe constraints on resources and many competing priorities, available resources have to be used as effi ciently as possible to address important challenges and goals, such as the Millennium Development Goals. Use of research evidence to inform decisions is crucial. As suggested by Hassan Mshinda, the Director-General of the Commission for Science and Technology in Tanzania: “If you are poor, actually you need more evidence before you invest, rather than if you are rich.” But neither the problem nor the need for solutions is limited either to health or countries of low and middle income. Expenditures and the potential for waste are greatest in high-income countries, which also have restricted resources and unmet needs, particularly during a fi nancial crisis. Having good evidence to inform diffi cult decisions can be politically attractive, as shown, for example, by the US Government’s decision to include US$1·1 billion for comparative research (including systematic reviews and clinical trials) as part of its $787 billion economic stimulus bill. To paraphrase Billy Beane, Newt Gingrich, and John Kerry, who have argued for a health-care system that is driven by robust comparative clinical evidence by substituting policy makers for doctors: “Evidence-based health care would not strip [policymakers] of their decision-making authority nor replace their expertise. Instead, data and evidence should complement a lifetime of experience, so that [policymakers] can deliver the best quality care at the lowest possible cost.” Lancet 2010; 375: 427–31