LDH, IES and the Reign of Frogs
Okay, barring the bad karma that seems to hang over the state of Illinois, Arne Duncan is now firmly ensconced as President Barack Obama’s nominee as Secretary of Education, thereby forestalling the Apocalypse predicted by the detractors of Linda Darling-Hammond. But eternal vigilance is the price of freedom, or something like that, and the fears have shifted to the future of the Institute of Education Sciences (IES), the federal government’s arm for education research and evaluation. Founding Director Grover “Russ” Whitehurst has moved on from his six-year term, and there are researchers lying awake at night in fear that President Obama might choose LDH, a one-time colleague of skoolboy at Teachers College, Columbia University, as his successor. I’m not entirely sure what they are afraid of, but clearly the dismantling of the research infrastructure built up over the past six years is near the top of the list.
If Linda Darling-Hammond is to serve in the Obama administration, skoolboy is not sure that the post of Director of IES would be the best use of her talents. But even if she were to be appointed to this post, I don’t think it would start raining frogs. Checks and balances on the actions of the IES Director abound, including the National Board for Education Sciences, which has the responsibility of approving the research priorities proposed by IES. Members of the National Board for Education Sciences are nominated by the President and confirmed by the Senate. The current chair of this board is Eric Hanushek, Senior Fellow at the Hoover Institution at Stanford University, and the vice-chair is Jon Baron of the Coalition of Evidence-Based Policy. And let’s not understate the difficulty of rapid change in a complex bureaucracy where most of the work is done by career civil servants, not by political appointees. (At least, that was my experience many years ago in a prior incarnation of IES. Working under Checker Finn, by the by.)
But the real problem is not with the organization of IES, which has made great strides in setting out criteria for fundable research, and implementing a rigorous peer review system that pushes ideological predispositions to the sidelines. (I serve on one of the IES standing review panels, along with a couple of hundred other researchers.) Rather, the problem is the failure of the education research community to develop a stockpile of effective educational interventions shown to work in multiple contexts. A quick overview of the kind of research that IES has been funding over the past four years tells the tale.
IES has adopted a progressive strategy, modeled on clinical trials in medical research, for funding education research to improve academic outcomes. Identification studies are intended to use existing data to identify existing programs and practices that are associated with better academic outcomes. A successful identification study will lead to a development project, in which a new education intervention (e.g., a new curriculum, a new instructional approach or program) is developed, and some preliminary data on effectiveness are gathered. Promising interventions are then evaluated in local settings using rigorous experimental and quasi-experimental methods designed to discern the efficacy of the intervention in those settings. Finally, those interventions demonstrating practically and statistically significant impacts on participants are “scaled up” – implemented more broadly in multiple settings, with multiple groups of participants, and without the direct involvement of the intervention developers in the replication sites.
Of the 275 regular education and special education research grants awarded by IES in research competitions between 2004 and 2008, only 7 – less than 3% – were scale-up projects, and an additional 26% were efficacy projects. The vast majority were development and identification projects that lacked prior evidence of program impact. Such projects are the raw material that might eventually lead to interventions that work at scale; but many will not, and even for the promising interventions, it may be many years before we are confident that they work as intended. That will be true no matter who inherits the IES Directorship.