Federal Policy and the Role of Research
Michelle and Jack continue their conversation about federal authority, paying particular attention to the role research should play in K-12 policy decisions.
Schneider: We've talked a lot about what the federal government shouldn't be doing. What in your mind should it be doing—either that it is already undertaking, or that it might in the future?
Rhee: I think much of the federal government's job in education is to set guidelines, establish accountability systems, and promulgate best practices. I think something that the Department of Education has been doing of late, which I think is important, is being a catalyst for states taking strong policy actions. I think Race to the Top sparked a lot of important conversations and actions in states. Probably more than the country has ever seen, which is a positive.
Also, I think there have been real efforts from the Department to try to incentivize innovative ideas and programs as well, like through the I-3 grants.
Are there things you think the Feds ought to be doing more of? Things you like?
Schneider: Beyond ensuring basic constitutional protections, I think the federal government is most effective when it builds the capacity of the states. Funding is one obvious way that it can engage in such work—evening out inequities that exist across localities.
But I agree with you that disseminating best practices is another way to go about the process. Still, as I've noted previously, I'd like to see the Department of Education use a lighter touch and show a bit more interest in research (though certainly not in so narrowly-defined a manner as has been the case with the relatively useless "What Works Clearinghouse"). In fact, I'd love to see them develop better ways for states, districts, and schools to engage with the vast stores of educational research that go mostly untapped. I think there's tremendous potential there, and that the Department of Education is extremely well-positioned to lead that effort.
Now, I wouldn't advocate returning to an early-20th century conception of the federal government's role in education, which primarily consisted of compiling reports. But I'm also wary of efforts to engineer social policy at too great a distance from the site where it is to be implemented. Not for any ideological reason. Just because it tends not to work. The reality on the ground tends to be too different from place to place, and that context-dependency leads to some pretty significant unintended consequences.
The trick, then, is to strike the right balance of engagement. For instance, while I think it would be a missed opportunity if the U.S. Department of Education didn't offer some guidance about something as critical as teacher training (a current topic of interest for them), I also think that it would be misguided for them to prescribe a particular program. So while I agree that an effort like Race to the Top generated conversation, I would also say that it errs too far in the direction of prescription.
Rhee: I agree 100% that the key is striking the right balance. Incentivizing and guiding states to adopt best practices without being prescriptive may be a tough needle to thread, but it's the right focus. I think incentivizing and guiding has to be done well. I'm not sure a better "What Works Clearinghouse" is enough. I mean, do people really look at that or pay attention? If not, what could the Feds do that would encourage more districts and schools to do so?
I disagree with your contention that Race to the Top erred too much on the side of being prescriptive. In fact, if you look at the breadth of how states and districts are implementing certain things (like teacher evaluations) I think there's a lot of diversity. Maybe even too much. In an ideal world, we'd have some way of capturing which new systems work the best and begin to hone states in on the best ones.
Schneider: As far as I know, nobody pays much attention to the What Works Clearinghouse. Partly that's due to their constricted definition of evidence, which has severely limited their findings. And partly it's a product of the fact that the Department of Education hasn't really promoted it. There just doesn't appear to be much interest in cultivating a research-oriented culture.
That's a shame, because in the abstract the WWC is a great idea that really leverages the utility of the federal government. Just imagine if the Department of Education was incentivizing and guiding states—I'm borrowing your language here—to adopt practices informed by research. And imagine further that the they were conducting serious research at the same time and channeling that information back to states and districts.
I'd love to see that approach applied to the rollout of something like Common Core, where instead of healthy skepticism and an orientation toward research, we instead see a lot of certainty. Certainty is never the basis for good policy.
It sounds, actually, like you agree with me. Though I would add that "capturing which new systems work the best" is going to be a context-dependent process. What may work best in one place, for one set of reasons, may not necessarily carry over to another place.
Rhee: I agree on several fronts. First, we should figure out how to get states and districts to adopt research-based practices and programs. I also agree that context sometimes matters, though I would add that it shouldn't be a reason for every jurisdiction to ignore best practices or research because they feel like need to create something that is uniquely "them."
Let's talk about your comment about "constricted definition of evidence." I think this is a tough one. How do we make the call on what there's evidence for? For example, you've repeatedly said in our blogs that you feel there's research showing VAM is flawed. It's true that there's evidence that VAM is not perfect. I'd argue there's also good research that shows that VAM, while not perfect, is one of the best tools we have. And that there's a strong basis for using it—as one of several measures—in teacher evaluations. So what are we saying is research-based and who determines it?
I'm also curious about your thoughts on Common Core. Are you saying the problem there is one of messaging?
Schneider: You're right; a lot of research indicates that VAM is better than many alternatives. But an even greater amount of research indicates that VAM is extremely limited in what it can account for, and that its results will vary significantly from year to year.
I don't think that makes a case for using value-added measures. Instead, I think the research indicates a need for better systems. As a short-term solution, I'm open to using a range of our least bad tools, as long as they are used for informational purposes rather than for hiring and firing decisions. But let's not for a moment allow ourselves to believe that we've found something that works.
As for Common Core, my point isn't that the messaging has been bad, though it has been. My point is that there should have been more piloting and more research before it became a major policy push. This is true for the standards themselves, though I think they're generally better than many previous state standards; and it is particularly true for tests like PARCC.
My advice for all these matters would be: go slow, be skeptical, rely on evidence, allow for error, and proceed with the greatest humility.
Rhee: On VAM, this is where things get tricky. You say "an even greater amount of research indicates that VAM is extremely limited in what it can account for, and that its results will vary significantly from year to year," which is true; but the fact of the matter is that what districts are using now is worse in terms of the limitations. Surely we can't wait for the perfect solution before we start using a better system for making personnel decisions. After all, we've been using worse systems for those decisions until now.
I agree with your thought that we can't be touting these changes and reforms as silver bullet solutions. We have to approach these shifts with the mindset that we need to constantly be improving them. That said, I want to push back on the notion of "go slow." I'd argue that we need to operating with a sense of urgency. We can't be reckless, but we also can't go slow.
Schneider: Look at the long history of unsuccessful reform efforts and see what they have in common. Time after time, the combination of certainty and haste has led to failure. And while this fact shouldn't preclude us from acting, it also can't be overlooked. Policymakers can choose to ignore that; but willful disregard for reality will do little to mitigate its consequences.