OMB Pushes for More Rigorous Program Evaluations
The federal Office of Management and Budget is increasing its push to get federal agencies to put their money where the research is.
In a memo to Education Secretary Arne Duncan and other agency leaders, the OMB has called for all fiscal 2014 budget proposals to include a separate section detailing the departments' "most innovative uses of evidence and evaluation."
Unlike the Program Assessment Rating Tool used to judge programs' budgetary worth—and often criticized for slow review cycles and findings of incomplete evidence—the memo calls for federal agencies to create and expand research partnerships to study programs, include cost-effectiveness calculations and embed the evaluation structure into program grants from the start.
In education, that could provide a big opening for the nation's regional educational laboratories, which in their most recent contracts were overhauled to include more partnering with state, local and other research groups.
The Education Department's Institute of Education Sciences, which oversees the labs, "wants us to develop research alliances and put people together around topics that are priorities for states," said Barbara Foorman, the first Commissioner of the National Center for Education Research and now head of the Regional Educational Laboratory for the Southeast Region. Foorman told me earlier this spring, as the lab got up and running, that in tight budget times, both the federal and state education departments are pressing for more research on programs: "They're looking for more evidence of effectiveness before they start writing checks."
The memo highlighted the Washington State Institute for Public Policy, which provides analyses of program effectiveness and return on investment for the legislature in that state.
OMB is also calling for agencies to look for quick, cheap and dirty approaches to the research itself. Traditionally, "gold-standard" randomized, controlled trials take five years or more and tens of thousands of dollars to conduct. To assist in cost-saving, the OMB has highlighted a brief by the Washington-based Coalition for Evidence-Based Policy suggesting ways to use the already-produced administrative data to create lower-cost, quicker-turnaround experimental evaluations.
"We strongly support this new effort," said Jon Baron, coalition president, in an email. "As demonstrated in fields such as medicine and welfare policy, such evidence-based approaches can greatly increase government's effectiveness in addressing critical national problems in social policy and other areas; and identify important opportunities for budget savings to help address the long-term deficit problem."
Moreover, the OMB memo hints that the Education Department won't be alone in using waivers to encourage states and districts to test out new approaches.
"One of the best ways to learn about a program is to test variations and subject them to evaluation, using some element of random assignment or a scientifically controlled design," the memo says. "OMB invites agencies to explain how they will use existing waiver authorities to evaluate different approaches to improving outcomes."
It also encouraged federal agencies to follow the lead of the labor and justice departments' "pay for success" models, in which private groups "invest" in promising interventions to solve social problems and are repaid by federal grants in return for showing progress.