What Do We Mean by "Proven" Programs in Education?
One of the greatest impediments to policies promoting the use of proven programs is a lack of agreement about the criteria for "proven." Among policy makers, the likelihood that they will have to preside over endless battles among academics on this question makes them want to forget about the whole thing.
Fortunately, a consensus definition of "proven" is beginning to solidify. It is best stated in the standards for scale-up grants under the U.S. Department of Education's Investing in Innovation (i3) initiative, which demands at least one large, randomized evaluation showing positive effects of a program on outcomes of importance, or at least two large matched studies. That is, students in schools using given programs and very similar schools using ordinary methods are pre-tested and post-tested to see if students in schools using the program make greater gains. In randomized studies, schools or teachers are assigned at random to use the program or not (the "Gold Standard" of experimental design), and in matched studies, schools or teachers who chose to use the program are compared to similar ones who did not. Standards being developed by the Annie E. Casey Foundation add to these common-sense standards requirements about acceptable study durations, measures, and other features, and similar standards are used by our Best Evidence Encyclopedia (www.bestevidence.org).
What is important about these standards is that they are relatively straightforward to apply, and similar standards are being used in all areas of children's services (e.g., delinquency prevention, social-emotional learning, and parent training). Not that every academic agrees with these standards; many reject out of hand the entire idea that any quantitative education measure measures anything. Yet there is enough consensus among those academics concerned with policy to make the standards useful.
Although evidence standards are sure to be debated and to evolve over time, there is enough agreement today to make it possible for government to encourage schools to use programs that meet these standards and to focus on helping program developers meet them. Improvements in educational programs and student learning can't wait while we argue about exactly what standards to use, and now they don't have to.
NOTE: Robert Slavin is co-founder of the Success for All Foundation, which received a $49 million i3 scale up grant from the Department of Education in 2010.