UPDATED: Some Academics Push Back on Teacher-Student Link in 'Race to the Top'
It isn't just the teachers' unions that are nervous about the draft guidelines for the Race to the Top.
I've been making my way through the thousands of Race to the Top comments, and there are a handful from some academics who argue that there isn't a strong enough research base to support the use of "value added" data for decisions involving teachers.
The inclusion of such measures in the Race to the Top guidelines appears to fly in the face of the Obama administration's promises to fund research-based approaches in the Race to the Top, these scholars contend.
Helen Ladd, a scholar at Duke University, argues that the proposed criteria would lead teachers to focus even more heavily on standardized tests, and that the value-added measures themselves are not reliable.
"Even the most sophisticated approaches typically cannot distinguish the contribution of teachers from the classroom context, and they generate estimates of a teacher’s quality that jump around from one year to the next, largely because of the small sample sizes for individual teachers," she writes.
Ladd has been involved in a number of studies associated with this research group at the Urban Institute that makes use of value-added data.
Two scholars at the Economic Policy Institute, Sean Corcoran and Joydeep Roy, reiterate a lot of the methodological issues, add that teacher effectiveness can be affected by school culture and peers (see this story for more), and contend that "teacher effects" don't seem to persist after one year. They add that there isn't a lot of data to show how such value-added data would be used to improve teacher effectiveness or instruction.
Paul Barton, a former director of ETS' policy-information center, has issues with the quality of tests now in place, the definition of growth (what does a year of growth or one grade level mean, anyway?), and the ability of value-added methodologies to separate out teacher effects from other types, such as the orderliness of schools, the adequacy of materials, instructional programs, leadership, and so forth.
"It is, I believe, an impossible task to separate the effects of each of these elements so as to assign quantitative values to each, and use this for accountability or to control for them to isolate teacher effects," he writes.
And the folks at the American Educational Research Association say that the use of test scores for teacher accountability contravenes prescribed standards in the measurement field for the use of assessments.
There is no doubt that this is a highly complicated area. The solution, if there is one to be had in this field, is that the value-added data should be incorporated with other measures of teacher effectiveness.
That's essentially the position of the teachers' union in Wisconsin, which has supported the undoing of a law in that state that blocks a teacher-student data link. My understanding is that it's also the position of the Obama administration that evaluations should consist of multiple measures.
In this blog item, I wrote about how that might be accomplished: by looking at the value-added data only at the top and bottom quartiles of effectiveness, over several years, and by incorporating information from teacher observations based on a solid set of performance standards. (Think Charlotte Danielson's framework or those used by Teach For America or those that ground the national-board certification process.)
Perhaps there is something to be learned from the Teacher Advancement Program, which uses value-added data as part of its system, and, I'm told, is doing some work to figure out whether observational ratings of teachers correlate to the value-added scores.
As a national policy agenda, though, this is only going to work if teachers think that it's doable. So if you're a teacher, what do you think? Write in and tell us.
UPDATE: Don't forget to check out Debbie Viadero's recent story on methodological issues with teacher value-added.