« Performance-Based Assessment and Teachers | Main | Teacher Beat Celebrates 1st Birthday! »

Peer-Assistance and -Review: The Toledo Numbers

| 1 Comment

There was a bit of a mini-controversy in June when the New Teacher Project released its Widget Effect report.

But it wasn't the report's overall thrust that did it. Pretty much everyone agreed that our current systems for evaluating and offering assistance to struggling teachers are crummy.

The controversy was about the data on dismissals in one particular district: Toledo, Ohio. According to the district's personnel records, Toledo dismissed one tenured veteran and did not renew five novice teachers' contracts, the NTP reported.

But what about the district's much-heralded Peer-Assistance and -Review model, a number of sources wrote me afterward. Aren't the dismissal numbers much higher than that? Most people were merely confused, but some accused the NTP of willfully skewing the data.

And the American Federation of Teachers put out this press release:

"While the overarching conclusions of the report are sound, we have concerns about the report's data, particularly with respect to teacher evaluations in Toledo, Ohio. Toledo has a highly regarded teacher evaluation system ... that produces much better results than those described in this report."

So what gives? Well, I've been talking to folks on both ends and I've started to do some digging around the numbers. To put it kindly, the data-gathering is really a mess. What it boils down to is that the district and the Toledo Federation of Teachers had entirely different ways of "coding" dismissals through the PAR system.

For instance, the district doesn't appear to record teachers who resigned after being in the program as having been dismissed, but the union counts them as such. Also, it appears that the PAR program applied to some long-term substitute teachers. These teachers wouldn't have necessarily shown up in personnel records, but the union may have counted them within the overall PAR figures. And finally, the union used the term "terminated" teachers to refer both to nontenured teachers who were so poor that they were let go before their contracts were up, and to tenured veterans who were put into PAR and ultimately dismissed. The district, by contrast, separated out tenured from non-tenured teachers.

There's a lesson here for any district or union that wants to try peer review: Agree on common definitions and change your data systems accordingly, or it will be really hard to justify your numbers.

OK, I know you want the actual figures. Well, I'm not going to publish what I've got for a couple of reasons. First, my information isn't complete, and second, it's my understanding that officials from TFT and NTP are working together on trying to audit the numbers and come up with figures they can agree on. I don't want to insert myself in that process.

But count on the fact that we'll be bringing you updates once we know what's what.

Does the Toledo PAR really produce "much better results" than other evaluation systems, as AFT has asserted? We'll see.

1 Comment


Since I've been griping at you lately, I need to thank you for your research.

I'm not surprised with your findings. My data was provided in an old fashioned booklet that wasn't on the Web.

You are also correct that now-a-days systems must nail down their definitions and methodology at the beginning. But I'm also nostalgic for the time when bargains were sealed with handshakes.

That being said, I just reread the Tulsa application for a $55 million grant for data-driven evaluuations. It calls for a VAM model to be developed by the top vendor.

School reform requires trust. I can't visualize any situation where I'd trust that much. Of course, I think that that just means we have to make peer review work. Even if the TNTP, which is involved in Tulsa also, is acting in good faith and could answer all of my questions, I can't see how we can gamble so much on our ability to invent new VAM models in the next few months.

Comments are now closed for this post.


Most Viewed on Education Week



Recent Comments