When low-performing teachers were replaced under Washington’s teacher-evaluation system, student achievement markedly improved, a new study concludes.
The study seems to bolster the theory of action behind D.C.'s IMPACT program, which was designed partly to weed out poor teachers and replace them with better ones. But that doesn’t mean other districts should rush to adopt this approach: D.C. had a flush supply of talent able to step into these positions, and many districts aren’t that lucky.
The study, by Melinda Adnot, Veronica Katz, and James Wyckoff, all of the University of Virginia, and Thomas Dee of Stanford University, was released as a working paper Jan. 25 by the National Bureau of Economic Research.
Since first established in 2009, Washington’s evaluation system, called IMPACT, has been a lightning rod. The system couples multiple observations and test scores to evaluate each teacher.
Teachers have complained that IMPACT is confining and that it’s harder to get good scores in schools where most students face poverty. And observers worry that IMPACT’s considerable pressures could ultimately lead to higher rates of teacher turnover. (Research generally indicates that teacher turnover hurts student achievement, probably because it affects the overall culture and morale in a school building.)
For the study, the researchers used a quasi-experimental approach to look at the link between teachers and student achievement from the 2009-10 to the 2011-12 school years.
They found that when teachers identified as low performing left or were fired, and were subsequently replaced, student achievement jumped significantly, by about a fifth of a standard deviation in math. The effect in reading was less pronounced, about .14 of a standard deviation.
Nearly all the turnover among low-performing teachers was at high-poverty schools, the study noted.
“By comparison to almost any other intervention, these are very large improvements that are situated among some of the neediest students,” the researchers wrote.
When high-performing teachers left, on the other hand, their students’ scores fell slightly—probably because it was hard to get a replacement of similar caliber—but those effects weren’t statistically significant.
The annual turnover rates in Washington over this period, at 18 percent, were higher than the 8 to 17 percent found in other urban districts, but overall benefited students, the study concludes.
The study cautions, though, that D.C. had a unique context in which it was able to fill vacant positions with good talent—one that not every district can match.
For example, the scholars write: “Critically, the supply of entering teachers [in D.C.] appears to be of sufficient quality to sustain a relatively high turnover rate.” And they note that the specific context of D.C.'s labor market, possibly coupled with the district’s cash bonuses for high-performing teachers, might have helped keep this pool strong. But other school districts without those features could see “different results” from a similar system.
Translation: Your mileage may vary.
It makes some sense, when you consider that D.C. is a place where a lot of young teachers want to be. On the other hand, if an evaluation system creates a lot of churn and isn’t balanced out by being able to attract high-performing talent, then it likely wouldn’t have these results, and could even be counterproductive.