-Steve Rivkin, Rick Hanushek, and John Kain (2005)
"Having a top-quartile teacher rather than a bottom-quartile teacher four years in a row would be enough to close the black-white test score gap."
-Robert Gordon, Tom Kane, and Doug Staiger (2006)
"There are big differences in the amounts and kinds of learning that different teachers help produce....these effects are cumulative."
- Kati Haycock, Education Trust
It's everyone's favorite sound bite: good teachers alone can close racial and socioeconomic achievement gaps. But if the entire teacher effect doesn't persist from year-to-year - that is, a student only retains some fraction of the learning advantage they get from having a highly effective teacher - these claims simply don't hold up.
In a new paper, "The Persistence of Teacher-Induced Learning Gains", Brian Jacob, Lars Lefgren, and David Sims estimate how much of the teacher effect fades out over time. It turns out that kids lose more of these short-term test score gains that we (or at least I) thought:
"Our estimates suggest that only about one-fifth of the test score gain from a high value-added teacher remains after a single year. Given our standard errors, we can rule out one-year persistence rates above one-third. After two years, about one-eighth of the original gain persists."
Yes, you read that correctly. Even if you rely on the upper bound estimates of teacher effect persistance from this study, only a third of that gain sticks around. If you take their point estimate, only 20% of this gain persists. If gains fade out at this rate, we may be overstating the ability of highly effective teachers to contribute to students' long-term academic skills, says Jacob:
"Our results indicate that contemporary teacher value-added measures may overstate the ability of teachers, even exceptional ones, to influence the ultimate level of student knowledge since they conflate variation in short-term and long-term knowledge. Given that a school’s objective is to increase the latter, the importance of teacher value-added measures as currently estimated may be substantially less than the teacher value-added literature indicates."
Jacob and colleagues conclude that we should revisit the "5 great teachers can erase gaps" claim that is so common in education policy discourse:
"Previous researchers have referenced a counterfactual world in which a series of high value-added effects for a hypothetical student with a string of good teachers may be simply added together. Given this scenario, researchers and policymakers have advocated the widespread use of such value-added measures in a variety of education policies including teacher compensation and teacher/school accountability. Our results suggest some caution should be taken in focusing on such measures of teacher effectiveness. If value-added test score gains do not persist over time, adding up consecutive gains does not correctly account for the benefits of higher value-added teachers. Of course, the same caution should be attached to any educational intervention. Hence, the broader implication from this work is that researchers and policymakers should make greater effort to track the long-run impact of education policies and programs."
If you can't access the paper, I've linked to Brian Jacob's contact info above, or shoot me an email (eduwonkette (at) gmail (dot) com).
Update: * To clarify, this paper does not find that teachers "don't matter." If every teacher moved students forward 2 grade levels - an effect twice as large as the gains we expect teachers to produce - we would find no "teacher effects" on test scores, i.e. having one teacher versus another wouldn't matter because all teachers would be equally effective in increasing test scores. But teachers would still make enormous contributions to students' learning in this scenario - they still "matter." Jacob and colleagues' point is simply that the difference between having a below and above-average teacher may be inflated in the current literature because we've been focusing on short-term versus long-term gains.
* Chad Aldeman at The Quick and the Ed misunderstands the implications of the paper: "These findings in no way challenge previous studies indicating teacher effects accumulate over time." This study does find that teacher effects accumulate - a student does, after all, hold on to 20% of the teacher-induced learning gains - but they do not accumulate in the additive way that those quoted at the top of this post have suggested. The above quotes assume that students carry forward the full gain, and that as a result, we can close the achievement gap by giving students five 84th percentile teachers in a row. If teacher-induced gains decay at the rate documented in this paper, this sound bite does not hold up.
In mathematics many of us believe the inverse, that is, one bad teacher before high school can leave a student in poor shape for a long time, in a way that good or reasonable teachers cannot easily undo.
Bad here, though, doesn't mean some percentile rank, it means uncomfortable with content.
I know what will really advance the profession--getting the argument out there that good teachers don't matter!
I'll try my hand at reading the math, but now you mention it, your analysis makes sense. I'm constantly having students come back and thanking me, but I've never had an alumni come back and thank me for raising test scores. We discuss what they learned in our class, what they remember, and how it was relevant. Not surprisingly, my former students retain a way of looking at the world, of looking at school work, an interest and motivation etc. Its always fun to hear about the specifics they retained and why, but I don't doubt that standardizing testing improvements would fade.
I also know that some of the things we learned in class are discussed afterwards on the street corner as homeboys are hanging out. I wish they had left the gang, but of course I'm incredibly proud of that compliment.
But getting back to the teacher quality argument, under what scenario would you find inner city neighborhood schools where students would get four or five warm bodies in a row? Has anyone ever heard of a high poverty neighborhood school where you could get four competent teachers in a row? In my experience, year in and year out, regular students find that about 1/2 of their classes are functional. In many cases, those classes have a good teacher, but given the circumstances, they aren't close to effective.
But I've got news for the left and the right wing true believers. If they could coerce districts into coercing effective teachers to transfer from effective schools, we would still end up the same. Those effective teachers would bail out immediately, unless we did something about chronically chaotic schools. Its the dispect shown to teachers, and the inability of the system to let teachers teach, that has driven so many great young teachers out of high poverty school.
Its such a shame, because if you can do it and maintain an emotional equilibrium, teaching in high poverty schools is the greatest job in the world. Show some respect for teachers and we could provide four or five effective teachers in a row. Until we can address systemic realities, though, the best we can do it just get rid of the most ineffective teachers even if we can't find more that a Long Term Sub to fill in.
John #1 - If the post was unclear, I apologize - but the finding of this paper is not that good teachers don't matter. Even if students don't take forward the entire gain provided by a highly effective teacher, they are still better off having an above rather than a below average teacher. It's simply that the size of these effects on test scores may be smaller than we previously thought - and getting a better read on these effect sizes is important when we're making claims about closing gaps. And as John T. points out above, teachers "matter" in lots of other ways as well.
John T. - You make an important point - that the contexts in which teachers teach enable and constrain good teachers. Plugging in exceptional individuals will never be enough - something that a lot of the teacher quality debate misses.
Jonathan - I like the way you flip this question around.
I've always been surprised that people are so quick to label teachers as "good" or "bad" without considering their teaching environment. As John T. pointed out, a good teacher in a bad teaching situation may not create a stellar outcome. Also, teachers who excel in one type of school are not necessarily going to excel in all types of schools.
For instance, I found that in inner city schools, a highly academic teacher, who would be perfectly suited for teaching AP Physics in the 'burbs, may not have as much success with the kids as a teacher with no graduate degree but with great classroom management skills. A PhD isn't going to do much good if the kids won't sit still for the lesson!
As eduwonkette noted, serious socio-economic problems bedevil students in low income schools -- problems that even the "best" teachers aren't going to be able to fix by themselves.
It's tough to say precisely what a good teacher is, particularly when your sole measure is test scores. A social studies teacher of my acquaintance tells me in his previous school, about 30% of his students passed the Regents exam, and that he had the highest passing rate in the building. In our school, his passing rate is closer to 90%.
He swears his methodology hasn't changed at all.
NYC: Good point. Your info supports my theory that it's difficult to label an educator as "good" or "bad" independent of his or her school setting.