New York City’s now defunct schoolwide performance-pay program didn’t increase student learning in any statistically meaningful way, according to a new study by Harvard University economist Roland Fryer. And a large majority of the schools decided to allocate their pay bonuses to all teachers evenly, rather than using some other set of criteria to recognize specific teachers’ contributions.
The study looks at the three years of the program, which ran from 2007 through 2010. It uses a randomized experiment with about 200 schools, comparing the results from a “treatment” group of schools participating in the pay plan to a “control” group that didn’t implement it.
The findings may sound familiar: They’re similar to an earlier study that drew on the school randomization Fryer helped set up for research purposes. Teacher Beat reported on that study back in February. The program required a team at the school to allocate awards for hitting performance targets, and since you can read all about those details in the prior item, I won’t repeat it here.
Other findings from this latest study: The program didn’t raise test scores at all and may have slightly depressed middle school scores in the participating schools. The impact of the incentives on student attendance, behavior, course grades, regents test scores, and high school graduation were negligible, Fryer writes. And it did not seem to affect teacher behavior either, as measured by retention rates in the school or the district; absenteeism; or teacher perception of the learning environment.
Finally, more than 80 percent of school teams rewarded the same bonus amount to nearly all the teaching staff in the schools, meaning that in practice these bonuses operated less like a performance pay system and more like a simple raise.
The paper includes a discussion of possible explanation for the lack of effects. One hypothesis Fryer lays out is that the incentive scheme, like others in the United States, may have been too ambiguous in its goals and complex in its means to effect a change in teachers’ behavior.
One important addition: This was a costly experiment for New York City taxpayers, for several reasons. The program itself was expensive, costing on the order of about $75 million over its duration. And it will continue to be costly, because the program, put in place in 2007, was made as part of a deal with the United Federation of Teachers. The district’s tradeoff was in allowing teachers who agreed to pay a minimal amount toward their pensions to retire with full benefits five years earlier.
Of course, there have been moves to curb public pensions lately. But generally that’s a tall order for poliycmakers, partly because pensions are legally viewed as property in some state laws. The moral of the story: Performance pay is temporary, but a pension is pretty much forever.