Guest post by John Thompson.
Elaine Weiss’ ”Mismatches in Race to the Top Limit Education Improvement” and the General Accounting Office’s (GAO’s) “Race to the Top: States Implementing Teacher and Principal Evaluation Systems Despite Challenges” make a fine set of bookends for the considerable evidence documenting the problems with the RttT. In the sections that deal with the experimental evaluation systems that were imposed on teachers and principals, an objective and careful reader would be hard pressed to tell from the evidence presented whether it was Weiss or the GAO who was explaining the RttT’s mistakes. After all, their basic methodology, of reviewing the written record and interviewing state, local, and federal officials, was the same. Weiss simply supplemented those resources with the work of a far greater number of scholars and journalists.
For instance, it was the GAO, not Weiss, who presented this indictment of the RttT’s intrusion into teacher evaluation,
Officials in 11 (of 12) states said teacher concerns about the scale of change, such as the use of student academic growth data and consequences attached to evaluations, challenged state efforts. State and district officials also discussed capacity challenges, such as too few staff or limited staff expertise and prioritizing evaluation reform amid multiple educational initiatives. Officials in 10 states had concerns about sustaining their evaluation systems.
It would not have been appropriate, however, for the GAO to explain why it is unlikely that those teachers’ concerns can be adequately addressed, or analyze the multiple, more promising initiatives that the RttT undermines. The GAO was not tasked with reading public comments issued before the RttT was authorized. The GAO was not tasked with reading the warnings of the National Academies of Sciences and the Board on Testing and Assessments of the National Research Council that predicted the problems inherent in the RttT’s design. It was not the task of the GAO to summarize those warnings that the value-added models being pushed by the RttT will be “unable to provide objective comparisons between teachers who work with different populations.” Finally, it was not the job of the GAO to explain in detail how value-added evaluations undercut the chances of the RttT’s push for college readiness standards.
Similarly, the GAO is not the institution that should summarize the National Academies of Sciences’ expert warnings that were ignored by Secretary of Education Arne Duncan when he “incentivized” top-down micromanaging of diverse types of schools throughout a huge nation. The GAO must leave it to others to cite the eminent scholars’ warnings against “tests that mimic the structure of large scale, high stakes tests” in regard to their lack of reliability, their unintended consequences, and in undermining the transition to higher-quality assessments. It is the job of experts like Weiss, educators, and citizens to recall the social scientists’ warnings in regard to the lack of technical capacity and the can of worms that would be opened when using the RttT approach to evaluate teachers of untested subjects.
Weiss’ report addressed issues other than the teacher evaluation systems encouraged by the RttT, but when it comes to the problems with mandating the use of student performance growth in evaluating educators, the prime difference between the two studies is the way they characterized the RttT process. Weiss described the implementation of the evaluation policies as a “bumpy road,” while the GAO said implementation has had “mixed results.”
Even this small distinction is largely due to a definitional nuance. The GAO considered the evaluation promises to be fully implemented “even if not all assessment measures within the major components were used for every teacher or principal. We also considered evaluation systems to be fully implemented even if they were not used to inform personnel decisions.” A similar qualification was applied to the six states that are still piloting the system. In other words, the toughest questions - including dilemmas where viable solutions are unlikely - have still not been tackled.
The GAO reported that up to 3/4ths of teachers do not teach tested subjects, so the RttT requires the development of Student Learning Objectives (SOLs). It found that 3/4ths of the states reported problems with that process. The GAO, like Weiss, cited Tennessee as a state where their solution is problematical. Teachers are evaluated using students who they have not taught and they can merely choose to be evaluated on subjects that they do not teach but where students score well.
The GAO cited additional problems, such as small schools where teachers teach multiple subjects. It is practically impossible for them to devise SOLs for all of those subjects. And, in such schools, the test results of one student can have a major effect on a teacher’s evaluation.
The GAO report of the problems with evaluations and principals was nearly as tough in indicting the RttT. It explained the teachers unions’ concern that principals will feel pressured to evaluate teachers too harshly. A principals union voiced the concern that some of their colleagues would be too lax in their teacher evaluations in order to make their schools look better.
North Carolina was cited as a state where these contradictory dynamics call the RttT’s approach into question. On one hand, principals and assistant principals will be evaluated with the same rubric even though they have different roles. A different problem is created by principals receiving the previous year’s value-added reports while evaluating teachers’ current practice. How will those principals not be biased by the previous year’s scores?
The most ironic capacity problem with principals was reported in New York. The GAO reported that “the time commitment required for observing and evaluating teachers prevented some principals from thoroughly reviewing evidence submitted for evaluations or providing meaningful feedback to teachers.”
School leaders were tight-lipped about the costs of litigation which accompanied the RttT “reforms.” The GAO was more explicit in documenting the costs of implementing the RttT’s approach to evaluations. One district had to spend three times as much money implementing the system than it received in RttT funding. Of the 12 states studied by the GAO, 10 reported that they will not be able to continue to pay staff who work on RttT programs after its funding ends.
As the National Academies of Sciences predicted, implementing RttT-favored evaluations undermined the capacity of districts to plan for the very different instruction and assessments of Common Core. The GAO reported, “District officials in New York and Maryland told us that their evaluation reform efforts took precedence over other initiatives, such as implementation of the Common Core curriculum.”
The GAO, of course, could not say it, but my reading of its report indicates that the following may be the best news about the RttT. The GAO reported, “A 2011 survey conducted by the New York State Council of School Superintendents shows that 81 percent of responding superintendents were concerned that cost considerations might prevent their districts from soundly implementing new evaluation requirements.” Reading between the lines, it sounds like the RttT is destined for the long list of abandoned educational silver bullets. And when the highest-profiled test-driven evaluations schemes in the states with the best funding are put on the back burner, what happens to those policies in states that did not share in the RttT bounty?
And that brings us back to the big difference between the reports of the GAO and Elaine Weiss. Weiss, properly, asked the questions about the “opportunity costs” of the RttT. She asked what superintendents and other leaders must ask; how can educators attempt to comply with distractions like the RttT and still find the time and money necessary to implement more promising reforms?
As long as the RttT money is coming through the door, intense federal Department of Education oversight will come with it. Education leaders will thus have to focus on complying with RttT regulations until the RttT funds dry up. Until then, administrators will be more accountable for the implementation process than real results. As we start evaluating the effectiveness of the RttT in raising student performance, however, the issues will change. It is hard to see how anyone who has read Weiss’ and the GAO’s evaluations of the process of implementing the RttT’s approach to teacher evaluations would expect that future studies will find much to praise in terms of real improvements in teaching and learning.
What do you think? Why did the RttT designers ignore social science? Why do they not listen to education experts? Will the Duncan administration listen to the GAO?
John Thompson was an award winning historian, with a doctorate from Rutgers, and a legislative lobbyist when crack and gangs hit his neighborhood, and he became an inner city teacher. He blogs for This Week in Education, the Huffington Post and other sites. After 18 years in the classroom, he is writing his book, Getting Schooled: Battles Inside and Outside the Urban Classroom.