eduwonkette_header_515.jpg

Through the lens of social science, eduwonkette takes a serious, if sometimes irreverent, look at some of the most contentious education policy debates. (Find eduwonkette's complete archives prior to Jan. 6, 2008 here.)

« Still a Bobo in Paradise | Main

Everyone's Favorite Sound Bite About Highly Effective Teachers Put to the Test

talking-mouth.gif
"By our estimates from Texas schools, having an above average teacher for five years running can completely close the average gap between low-income students and others."
-Steve Rivkin, Rick Hanushek, and John Kain (2005)

"Having a top-quartile teacher rather than a bottom-quartile teacher four years in a row would be enough to close the black-white test score gap."
-Robert Gordon, Tom Kane, and Doug Staiger (2006)

"There are big differences in the amounts and kinds of learning that different teachers help produce....these effects are cumulative."
- Kati Haycock, Education Trust

It's everyone's favorite sound bite: good teachers alone can close racial and socioeconomic achievement gaps. But if the entire teacher effect doesn't persist from year-to-year - that is, a student only retains some fraction of the learning advantage they get from having a highly effective teacher - these claims simply don't hold up.

In a new paper, "The Persistence of Teacher-Induced Learning Gains", Brian Jacob, Lars Lefgren, and David Sims estimate how much of the teacher effect fades out over time. It turns out that kids lose more of these short-term test score gains that we (or at least I) thought:

"Our estimates suggest that only about one-fifth of the test score gain from a high value-added teacher remains after a single year. Given our standard errors, we can rule out one-year persistence rates above one-third. After two years, about one-eighth of the original gain persists."

Yes, you read that correctly. Even if you rely on the upper bound estimates of teacher effect persistance from this study, only a third of that gain sticks around. If you take their point estimate, only 20% of this gain persists. If gains fade out at this rate, we may be overstating the ability of highly effective teachers to contribute to students' long-term academic skills, says Jacob:

"Our results indicate that contemporary teacher value-added measures may overstate the ability of teachers, even exceptional ones, to influence the ultimate level of student knowledge since they conflate variation in short-term and long-term knowledge. Given that a school’s objective is to increase the latter, the importance of teacher value-added measures as currently estimated may be substantially less than the teacher value-added literature indicates."

Jacob and colleagues conclude that we should revisit the "5 great teachers can erase gaps" claim that is so common in education policy discourse:

"Previous researchers have referenced a counterfactual world in which a series of high value-added effects for a hypothetical student with a string of good teachers may be simply added together. Given this scenario, researchers and policymakers have advocated the widespread use of such value-added measures in a variety of education policies including teacher compensation and teacher/school accountability. Our results suggest some caution should be taken in focusing on such measures of teacher effectiveness. If value-added test score gains do not persist over time, adding up consecutive gains does not correctly account for the benefits of higher value-added teachers. Of course, the same caution should be attached to any educational intervention. Hence, the broader implication from this work is that researchers and policymakers should make greater effort to track the long-run impact of education policies and programs."

If you can't access the paper, I've linked to Brian Jacob's contact info above, or shoot me an email (eduwonkette (at) gmail (dot) com).

TrackBack

TrackBack URL for this entry:
http://blogs.edweek.org/cgi-bin/mt-tb.cgi/4185.

Comments

In mathematics many of us believe the inverse, that is, one bad teacher before high school can leave a student in poor shape for a long time, in a way that good or reasonable teachers cannot easily undo.

Bad here, though, doesn't mean some percentile rank, it means uncomfortable with content.

I know what will really advance the profession--getting the argument out there that good teachers don't matter!

I'll try my hand at reading the math, but now you mention it, your analysis makes sense. I'm constantly having students come back and thanking me, but I've never had an alumni come back and thank me for raising test scores. We discuss what they learned in our class, what they remember, and how it was relevant. Not surprisingly, my former students retain a way of looking at the world, of looking at school work, an interest and motivation etc. Its always fun to hear about the specifics they retained and why, but I don't doubt that standardizing testing improvements would fade.

I also know that some of the things we learned in class are discussed afterwards on the street corner as homeboys are hanging out. I wish they had left the gang, but of course I'm incredibly proud of that compliment.

But getting back to the teacher quality argument, under what scenario would you find inner city neighborhood schools where students would get four or five warm bodies in a row? Has anyone ever heard of a high poverty neighborhood school where you could get four competent teachers in a row? In my experience, year in and year out, regular students find that about 1/2 of their classes are functional. In many cases, those classes have a good teacher, but given the circumstances, they aren't close to effective.

But I've got news for the left and the right wing true believers. If they could coerce districts into coercing effective teachers to transfer from effective schools, we would still end up the same. Those effective teachers would bail out immediately, unless we did something about chronically chaotic schools. Its the dispect shown to teachers, and the inability of the system to let teachers teach, that has driven so many great young teachers out of high poverty school.

Its such a shame, because if you can do it and maintain an emotional equilibrium, teaching in high poverty schools is the greatest job in the world. Show some respect for teachers and we could provide four or five effective teachers in a row. Until we can address systemic realities, though, the best we can do it just get rid of the most ineffective teachers even if we can't find more that a Long Term Sub to fill in.

John #1 - If the post was unclear, I apologize - but the finding of this paper is not that good teachers don't matter. Even if students don't take forward the entire gain provided by a highly effective teacher, they are still better off having an above rather than a below average teacher. It's simply that the size of these effects on test scores may be smaller than we previously thought - and getting a better read on these effect sizes is important when we're making claims about closing gaps. And as John T. points out above, teachers "matter" in lots of other ways as well.

John T. - You make an important point - that the contexts in which teachers teach enable and constrain good teachers. Plugging in exceptional individuals will never be enough - something that a lot of the teacher quality debate misses.

Jonathan - I like the way you flip this question around.

I've always been surprised that people are so quick to label teachers as "good" or "bad" without considering their teaching environment. As John T. pointed out, a good teacher in a bad teaching situation may not create a stellar outcome. Also, teachers who excel in one type of school are not necessarily going to excel in all types of schools.

For instance, I found that in inner city schools, a highly academic teacher, who would be perfectly suited for teaching AP Physics in the 'burbs, may not have as much success with the kids as a teacher with no graduate degree but with great classroom management skills. A PhD isn't going to do much good if the kids won't sit still for the lesson!

As eduwonkette noted, serious socio-economic problems bedevil students in low income schools -- problems that even the "best" teachers aren't going to be able to fix by themselves.

It's tough to say precisely what a good teacher is, particularly when your sole measure is test scores. A social studies teacher of my acquaintance tells me in his previous school, about 30% of his students passed the Regents exam, and that he had the highest passing rate in the building. In our school, his passing rate is closer to 90%.

He swears his methodology hasn't changed at all.

Post a comment

Ground Rules for Posting
We encourage lively debate, but please, no profanity or personal attacks. By commenting, you are agreeing to abide by our user agreement.

eduwonkette

eduwonkette
E-mail me

The opinions expressed in eduwonkette are strictly those of the author and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Get RSS

Get eduwonkette delivered by e-mail. Enter your e-mail here:

Delivered by FeedBurner

Advertisement
Powered by
Movable Type 3.34

EW Archive