Value-added estimates of teacher effectiveness vary depending on the controls and statistical regressions chosen, points out a report released yesterday that uses the Los Angeles Times’ controversial teacher-rating project as a case study.
Using the same data set as the newspaper, the report from the National Education Policy Center re-runs the data using a different value-added model, and reaches quite different conclusions about which teachers are effective. According to its analysis of the data, 46.5 percent of teachers in reading stayed in the same quintile of effectiveness under the NEPC model as in L.A. Times’ model, while 60.5 percent teachers in math stayed in the same quintile.
The group says that the differences are due to the L.A. Times’ method, which it contends didn’t account for potentially important factors, like teacher “peer effects,” a longer history of student achievement, and additional school factors. The NEPC, which has generally been critical of value-added measures of teacher effectiveness, claims that this analysis undermines the L.A. Times’ project.
It also finds evidence that the value-added method couldn’t account for the non-random assignment of students to teachers, a problem researchers like Jesse Rothstein of Berkeley have also highlighted.
But L.A. Times officials said that the NEPC review didn’t acknowledge that the newspaper excluded some 5,000 teachers from its analysis for some of the reasons indicated in the NEPC study.
NEPC receives funding from a National Education Association-funded think tank, the Great Lakes Center. The L.A. Times reporter covering the NEPC report is the same one who spearheaded the paper’s series using value-added data. I’ll leave you all to start the requisite finger pointing.
There is clearly an awful lot of he-said-she-said going on here, so for another view, check Nick Anderson’s write-up in The Washington Post. He quotes several statisticians who note that there’s no research consensus about which value-added regressions, methods, and controls make for the “best” or most reliable, precise, and accurate form of teacher measurement.
The NEPC study makes this point, too: “Our sensitivity analysis indicates that the effects
estimated for LAUSD teachers can be quite sensitive to choices concerning the underlying statistical model.”
That’s a different issue from whether they should be used at all and to what extent, which has gotten most of the ink. But it’s an important thing to think about as more and more states and districts begin this work.
I’ve made this observation before, but it bears repeating: If value-added is to be a part of teacher evaluations—and not end up in the courts—then all parties involved, like school leaders, unions, teachers, and principals, are probably going to have to sort through all the various options and formally agree on the formulas to be used.