Two Cheers for Professor Pallas
This morning, Columbia University professor Aaron Pallas sounded a responsible (if hasty) retreat from last week's attack on DCPS. After writing last week that DCPS had seemingly used "preposterous" assumptions to adopt an "idiotic" teacher evaluation policy, Pallas wrote this morning in the Washington Post's "Answer Sheet" blog that, "I'm happy to hear that DCPS seems not to have botched the calculation of the value-added scores [by doing simple-minded subtraction]...even if this is what DCPS is telling teachers it's doing." Pallas also did a little legwork, reaching out to DCPS technical advisor Rick Hanushek and talking to DCPS data honcho Erin McGoldrick. He reports Hanushek's description of the DCPS IMPACT system "reassuring." Not the most graceful apologia, perhaps, but not bad in the world of Glenn Beck and Keith Olbermann.
Other than the pro forma complaint that "Hess is willing to take on faith the validity of whatever DCPS and Mathematica actually did," I find much to like and little to sweat in Pallas's newest effort. (As to Pallas's charge, I think RHSU readers know I'm too mean to take much on faith. But I do know and regularly talk to folks at Mathematica and DCPS, and that gives me substantial confidence in their handiwork on this.)
Pallas makes several sensible points about the sensitivity of value-added models and the importance that these receive appropriate scrutiny. He writes, "I still contend that teachers whose careers were placed in jeopardy by the results should have been notified of the methodology in advance, not after the fact." That certainly makes sense to me. He points out, "As we learned from the financial meltdown, there are serious risks in relying on technologies so complicated that hardly anyone...truly understands them." I think this is fair and worth keeping in mind. And he writes, "I reserve the right to be critical of the procedures if and when they are made public" and argues that "many technical and practical issues must still be worked out before value-added measures can be fairly used in high-stakes personnel decisions." The first is eminently fair and the second is a reasonable contention.
If professor Pallas had only just written this column in the first place.