A decade ago, Washington, D.C., became one of the first school districts in the nation to tie teachers’ job security and paychecks to student test scores—and new research shows that it’s working.
The teacher-evaluation system, known as IMPACT, was controversial and unpopular among teachers and their union when it was first implemented. Under the system, teachers who receive “ineffective” scores are subject to dismissal, and teachers who score “minimally effective” or “developing” scores could face dismissal if they don’t improve. Teachers who are rated “highly effective,” on the other hand, are eligible for financial rewards and professional opportunities.
Hundreds of teachers have been fired as a result of IMPACT. The system has since gone through several changes, but those core tenets remain.
Now, two new papers find evidence that lower-performing teachers are substantially more likely to voluntarily leave the D.C. school system than their higher-performing counterparts, and those lower-performing teachers are also more likely to improve. And when struggling teachers leave, student achievement improves. This new pair of studies builds on research published in 2015 and 2016, and finds that IMPACT is yielding long-term, successful results.
“We found that in many respects, teacher-evaluation practices in [the D.C. school district] seem to be working to the advantage of students,” said James Wyckoff, a professor of education at the University of Virginia and one of the authors of the studies.
A National Shift
High-stakes teacher evaluation has been one of the major policy shifts nationally over the past decade. The Obama administration rewarded states that tied evaluations to student test scores, and by 2015, 43 states required student-growth data in teacher evaluations. But since then, the financial incentives have ended, and now 34 states require student-growth measures, according to the National Council on Teacher Quality.
And other studies of teacher-evaluation reform from the 2010s have yielded disappointing results, including the Gates Foundation’s multi-million-dollar, multi-year effort aimed at making teachers more effective through evaluation reform and measures. That study, published last year, found that the reforms did not significantly increase student achievement.
“There’s a general sense that teacher evaluation, to put it bluntly, is over and done,” said Jessalynn James, a post-doctoral research associate at the University of Virginia and a co-author of the reports.
See also: Timeline: How Teaching Has Changed Over the Decade
But IMPACT continues to show positive results, despite changes in district leadership and adjustments to the evaluation system itself. For example, the district added student surveys to the evaluations and reduced the emphasis on test-based value-added measures.
Researchers found that lower-performing teachers continue to leave the district at higher rates than high-performing ones. And when minimally effective teachers stick around, their performance tends to improve.
On average, when low-performing teachers leave, they are replaced by teachers whose IMPACT scores are almost a standard deviation higher—and student learning in those classrooms goes up by about two months.
That’s not quite as stark as it was during the first few years of IMPACT, Wyckoff said. That could be because over time, the pool of low-performing teachers has declined, either because they improved or they left.
The researchers note that D.C. has a unique context, which may account for some of the results.
“You want to make sure you can replace [teachers who leave] with teachers who are better, because it doesn’t work to replace someone who is equivalent or worse,” Wyckoff said. “There is a pool of applicants who want to become teachers in DCPS who are, on average, pretty good. That condition probably doesn’t exist everywhere.”
When Good Teachers Leave
When a highly effective teacher leaves, however, she is typically replaced by someone with worse IMPACT scores, the study finds. At least in reading, student learning goes down by at least two months.
Attrition for teachers who are rated either highly effective or effective is 13 percent. Even though that’s a smaller share than the overall attrition rate, Wyckoff said losing these teachers is still costly for students.
But the teacher-evaluation system is not the reason why those high-performing teachers leave. In 2017, only 3 percent of highly effective teachers cited IMPACT as one of the top three factors for leaving—compared to in 2013, when 21 percent said that was one of their top reasons. Most of those who leave now say there was nothing their school or district could have done to change their decision.
Among the factors for leaving that the district does have control over, highly effective teachers most often cited a desire for increased behavioral or instructional support, more schedule flexibility, increased compensation or benefits, and improved and clearer growth and leadership opportunities.
Wyckoff said he hopes the district uses the evaluation ratings to better work to retain high-performing teachers, such as through additional career opportunities.
Teacher evaluation, he said, should be a tool for district leaders to support teachers as they work to improve.
“The puzzling part for me around the national discussion is, if we believe that teachers are as important to student outcomes, don’t we want to measure that variability and work with teachers [to help] make them get better?” Wyckoff said.
“Too often, we focus on the dismissal of teachers or the huge bonuses that some of these teachers get paid,” he continued. “We don’t spend enough time thinking about the ways we can support the development of teachers once we identify the specific skills they may be weak on. Can we think about teacher evaluation in that formative, supportive way where the culture is one where we’re in this together?”
Image via Getty, chart via James & Wyckoff