Everyone's Favorite Sound Bite About Highly Effective Teachers Put to the Test
-Steve Rivkin, Rick Hanushek, and John Kain (2005)
"Having a top-quartile teacher rather than a bottom-quartile teacher four years in a row would be enough to close the black-white test score gap."
-Robert Gordon, Tom Kane, and Doug Staiger (2006)
"There are big differences in the amounts and kinds of learning that different teachers help produce....these effects are cumulative."
- Kati Haycock, Education Trust
It's everyone's favorite sound bite: good teachers alone can close racial and socioeconomic achievement gaps. But if the entire teacher effect doesn't persist from year-to-year - that is, a student only retains some fraction of the learning advantage they get from having a highly effective teacher - these claims simply don't hold up.
In a new paper, "The Persistence of Teacher-Induced Learning Gains", Brian Jacob, Lars Lefgren, and David Sims estimate how much of the teacher effect fades out over time. It turns out that kids lose more of these short-term test score gains that we (or at least I) thought:
"Our estimates suggest that only about one-fifth of the test score gain from a high value-added teacher remains after a single year. Given our standard errors, we can rule out one-year persistence rates above one-third. After two years, about one-eighth of the original gain persists."
Yes, you read that correctly. Even if you rely on the upper bound estimates of teacher effect persistance from this study, only a third of that gain sticks around. If you take their point estimate, only 20% of this gain persists. If gains fade out at this rate, we may be overstating the ability of highly effective teachers to contribute to students' long-term academic skills, says Jacob:
"Our results indicate that contemporary teacher value-added measures may overstate the ability of teachers, even exceptional ones, to influence the ultimate level of student knowledge since they conflate variation in short-term and long-term knowledge. Given that a school’s objective is to increase the latter, the importance of teacher value-added measures as currently estimated may be substantially less than the teacher value-added literature indicates."
Jacob and colleagues conclude that we should revisit the "5 great teachers can erase gaps" claim that is so common in education policy discourse:
"Previous researchers have referenced a counterfactual world in which a series of high value-added effects for a hypothetical student with a string of good teachers may be simply added together. Given this scenario, researchers and policymakers have advocated the widespread use of such value-added measures in a variety of education policies including teacher compensation and teacher/school accountability. Our results suggest some caution should be taken in focusing on such measures of teacher effectiveness. If value-added test score gains do not persist over time, adding up consecutive gains does not correctly account for the benefits of higher value-added teachers. Of course, the same caution should be attached to any educational intervention. Hence, the broader implication from this work is that researchers and policymakers should make greater effort to track the long-run impact of education policies and programs."
If you can't access the paper, I've linked to Brian Jacob's contact info above, or shoot me an email (eduwonkette (at) gmail (dot) com).
Update: * To clarify, this paper does not find that teachers "don't matter." If every teacher moved students forward 2 grade levels - an effect twice as large as the gains we expect teachers to produce - we would find no "teacher effects" on test scores, i.e. having one teacher versus another wouldn't matter because all teachers would be equally effective in increasing test scores. But teachers would still make enormous contributions to students' learning in this scenario - they still "matter." Jacob and colleagues' point is simply that the difference between having a below and above-average teacher may be inflated in the current literature because we've been focusing on short-term versus long-term gains.
* Chad Aldeman at The Quick and the Ed misunderstands the implications of the paper: "These findings in no way challenge previous studies indicating teacher effects accumulate over time." This study does find that teacher effects accumulate - a student does, after all, hold on to 20% of the teacher-induced learning gains - but they do not accumulate in the additive way that those quoted at the top of this post have suggested. The above quotes assume that students carry forward the full gain, and that as a result, we can close the achievement gap by giving students five 84th percentile teachers in a row. If teacher-induced gains decay at the rate documented in this paper, this sound bite does not hold up.