« Reform, Transform, Or Just Hang On for Dear Life? | Main | The Income Gap Compounds the Achievement Gap in Schools »

Is the Race to the Top a Race to Test?


According to an article in the Los Angeles Times, California’s state senate is scheduling hearings to consider the abandonment of a policy that blocks the use of student test score data for the evaluation or compensation of teachers, in order to qualify for federal "Race to the Top" funding. California recently created a data system to manage all the standardized test data. The Education Code was revised to state:

(c) Data in the system shall not be used, either solely or in conjunction with data from the California Longitudinal Pupil Achievement Data System, for purposes of pay, promotion, sanction, or personnel evaluation of an individual teacher or groups of teachers, or of any other employment decisions related to individual teachers. The system shall not include the names, social security numbers, home addresses, telephone numbers, or e-mail addresses of individual teachers. (California Education Code Section 10601.5, section c)

This clause has caught the attention of Secretary of Education Arne Duncan, who says it makes the state of California ineligible for his $4 billion “Race to the Top” fund. He intends to direct this fund towards what he considers innovative practices, including paying teachers more if they increase their students’ test scores.

Unfortunately, this approach dovetails neatly with the past seven years of federal pressure on schools to use test scores as the primary means of measuring learning.

On the face of it, it sounds reasonable to evaluate a teacher based on how well his students have learned. When I applied for National Board certification, I provided videotapes of my students engaged in discussions, samples of their work, and further evidence of all they had learned from my instruction. I deeply believe teachers should be responsible for how well their students learn. But the devil is in the details of how we measure learning.

How is Student Growth Measured?

The primary means of measuring student growth goes under the name Value Added Model (VAM). Using this method, some school districts have begun to analyze a teacher’s performance by examining her students’ growth during the time she taught them. In some ways, this seems more reasonable than the current NCLB practice of comparing this years’ students with last years’, and expecting constant growth. However, a fascinating study was released in May that sheds some disturbing light on the flaws in this approach.

Princeton scholar Jesse Rothstein points out several key problems. To quote from the study summary:

Rothstein’s study focuses on the challenge of distinguishing a teacher’s contribution from pre-existing differences among students. Teachers do disparate jobs – some teach “gifted and talented” classes, some focus on students with limited English skills, and some work with students with special needs. If accountability and merit pay policies are to produce improvements in teacher quality, it is essential to ensure that teachers who get the “right” students who test well do not get unfair advantages, and that teachers who get the “wrong” students do not get unfair disadvantages. It will do no good, and may even cause harm, to implement a merit pay system that rewards teachers for working with gifted students and penalizes those who work with more challenging students.
The crux of any reform in the pay system is that it be fair. If teachers working with the most challenging students face even more pressure to raise test scores, and are punished unfairly when their students do not perform for a variety of reasons beyond the teachers’ control, that will drive down morale and boost turnover in these schools.

Rothstein came up with a brilliant means of proving just how unfair this system is. If students are not assigned randomly to classes, then a teacher might be rewarded or penalized based on who was assigned to their class – and the pre-existing condition of these students.

To show this, Rothstein develops falsification tests for the VAMs. (Falsification testing evaluates an assertion by asking whether it has implications that are known to be incorrect.) Specifically, he asks whether the VAMs imply that 5th grade teachers (for example) have effects on students’ 3rd and 4th grade test scores. This test exploits the fact that future teachers cannot have causal effects on past outcomes, so a method that successfully distinguishes causal effects from pre-existing differences among students should not find signs of such effects.
In fact, the VAMs currently used for teacher accountability indicate that 5th grade teachers have large effects on students’ 3rd and 4th grade achievement. This reflects systematic sorting of students into classrooms on the basis of past achievement, producing substantial dispersion of students’ 4th grade scores and score gains - the growth in scores between 3rd and 4th grade – across 5th grade classrooms. Sorting on past reading gains is particularly prominent, though there is clear evidence of sorting on math gains as well.

Just to explain this further, imagine three fifth grade teachers; Ms. Best, Ms. Good, and Ms. Worst. If I analyze the scores of the fourth grade students heading into these three classrooms, and find that in fact most of the highest scoring students wind up in Ms. Best's class, then I have uncovered a non-random, and therefore unfair distribution. This is exactly what Rothstein found.

I experienced this firsthand a few years ago, when one of my sixth grade math classes included six students who had been held back at least once, some of them twice. These students were much harder to move academically than others, and their concentration in that class made the group as a whole tougher to teach. A study released last year found that students in homes with domestic violence not only suffered academically, but also brought down the scores of their peers in the classes they shared.

This article in the San Francisco Chronicle reveals that as many as 40% of the children in some urban neighborhoods suffer from post-traumatic stress disorder as a result of the violence they have witnessed at home and in their neighborhoods. This has profound effects on how these students learn. These problems are not distributed randomly in society, and these students are not distributed randomly within a school.

These are not excuses, but very real challenges faced by urban educators every day. It does not mean we give up on these students, but it does mean we have to be careful to craft solutions that support rather than further stigmatize these schools.

Rothstein’s research also showed that teachers who were successful in raising student scores in the short term, through test preparation for example, did not necessarily have a lasting effect on their performance. This is hugely important. Our schools already suffer from an overemphasis on test preparation. If we actually tie teacher evaluations and pay to these scores, we are likely to deepen this emphasis, and our students will suffer in the long run.

In demanding a link between test scores and teacher pay and compensation, Duncan has chosen a rather poor vehicle for innovation. There is no question that teacher evaluation should be strengthened, and there is plenty of room for innovative approaches to compensation as well. But there is nothing innovative about more emphasis on test scores.

Way back in November of 2007, candidate Obama said,

And by the way - don't tell us that the only way to teach a child is to spend most of the year preparing him to fill in a few bubbles on a standardized test. Don't tell us that these tests have to come at the expense of music, or art, or phys. ed., or science. These tests shouldn't come at the expense of a well-rounded education - they should help complete that well-rounded education. The teachers I've met didn't devote their lives to testing, they devoted them to teaching, and teaching our children is what they should be allowed to do.

He was right about that.

Now Obama says that tying test scores to pay and evaluations will not result in teachers teaching to the test. I do not comprehend how this can be so.

He and Secretary Duncan seem to have very little imagination when it comes to finding out which teachers are doing a good job. They continually return to test scores as the essential measuring stick for teacher quality. Teachers know that this shortcut will lead to a dead end.

California State Senator Gloria Romero has called for hearings to eliminate the provision separating state testing data from evaluations and pay. Here is a link to California Senators/Assembly if you would like to share your views with them.

What do you think? Is it time to evaluate and compensate teachers using student test scores?


Personally, I believe that it IS POSSIBLE to have a fair system of teacher performance pay. Test scores COULD be PART of the pay system if they valued student GROWTH rather than achievement. However, if I can determine by examining whatever system of performance pay is put into place what teaching position I need to transfer to in order to receive the pay, the system is unfair.
I am currently a fifth grade teacher in a high-needs school. The most common reason I've been given to justify the sorting of students that you document is laudible: "to best meet their learning style" - in other words, it is a good thing to place students with the teacher who is most likely to meet their known needs. For a variety of good reasons, I usually receive a disproportionate share of students, particularly boys, with moderate to severe behavior issues that have greatly impaired their education thus far. I usually have more than one 13 year old and several twelve-year-olds(try to explain "no child left behind" to them!)in the predominantly Hispanic classes (two years of English education = language proficiency, right?) If a kid has received a label or ADHHHD or bi-polar, he's generally mine.
This "unfair" placement has always been OK with me; in fact, it has become somewhat of a calling to accept each group of challenging students, roll up my sleeves, and see what I can inspire and enable them to accomplish. However, the day I know my "calling" is going to result in a significant loss of earnings is the day I'll be looking for a transfer.
I just can't justify an even deeper sacrifice to my family than I am currently making. My family already accepts the many uncompensated hours that I routinely work outside of the school day, understanding that it is necessary given my current placement (I rarely have the kind of class that allows me to sit down, let alone cope with paperwork, during the day). My family is already well-aware that, in a high-needs school, I have more out-of-pocket costs that my friends at "easier" schools do. The same fundraiser that nets $65,000 at the school two miles from mine nets $1500 at mine. I have a friend who averages $1000 per year in Christmas gift certificates from parents at her school, a far-cry from the bag of Starbursts, 3 candy-canes, and a 4-piece box of chocolates that I received last year (it would have been a 5 piece but one was missing!). I try to be discrete with the all-to-frequent school purchases taken from my family's budget, and I do not wish to know how much they add up to each year. Yes, I do throw the evidence in a drawer for my husband to tally at tax time, which helps.
I know from experience that the uneven placement of students is a prime reason to base performance pay on student GAINS rather than achievement levels. The placements you describe also supports a model where achievement gains across a team of teachers is evaluated rather than the gains of individual teachers, but that does introduce the problem of equally rewarding the less-than-stellar teacher merely fortunate to be a member of a strong team.
The inequity of a single test being used to determine compensation is particularly apparent in high-needs classrooms like mine where the day-to-day performance for each student tends to be very erratic. For example, from my days in 4th grade, I remember well the student who came into the room on the morning of the oh-so-important Florida Writes test and kicked the metal trash can across the front of my room where it dented my desk with a satisfying metallic crash. He had just had a fight with his stepmother on the way to school. I delayed starting the test as long as I dared, attempting to calm him down and verifying that his bad mood was not sufficient grounds to excuse him from testing that day (he was not "sick" or meeting the short list of conditions that would allow that). He was unable to put a word on his test paper until midway through the 45 minutes allowed, when he suddenly "shook it off" and diligently wrote part of a page before time was called. I had many papers documenting his above-average writing ability but that didn't mitigate the poor score of "1.5" he received on that day.
So, is it time to evaluate and compensate teachers using test scores? Hmmmm.... I still think so, but I have a long list of "ifs" to be met. Such a system is doomed to fail, I think, unless teachers still in the trenches determine its design.

Few professions have been more ingenious and steadfast that teachers in resisting progress, as the preceding comments suggest. Since all ways of paying teachers have flaws, try asking what about the present system has brought Americans to the international forefront of student academic success. Few would claim that it has. Meanwhile, the atypical anecdote remains the crutch used by opponents of progress.

The fact that there are flaws in a proposal seems like the very best reason to oppose it. I do not believe the anecdotes presented are atypical at all, but are precisely what one encounters in the real world.

It is possible to come up with innovative ways to pay teachers -- take a look at the Alternative Teacher Professional Pay System in Minneapolis -- http://atpps.mpls.k12.mn.us/. But as Susan Bischoff points out, teachers need to be involved in the design if it is going result in real change. After all, the goal is to motivate teachers, isn't it?

Student test scores (gains or strictly achievement) is ok to use if there is a model that can take into account all of the other things that impacted the student on the few days of the test. Did that child have breakfast? (Maybe there wasn't any food at home.) Did she get enough sleep? (Perhaps she couldn't sleep after she heard gunshots outside and her neighbors yelling.) Did the high school girl get enough "seat time"? (Maybe was she out 40% or more days of school due to needing to be at home to take care of her infant.) Do the parents value education? (Or do they say "why does my daughter need algebra, she's not going to college anyway" when the teacher calls about a low test grade.)

The value of education is a big one. Until our society as a whole sees the value of education and makes education a priority (and not just a priority in the rhetoric of politicians), we will continue to fall behind compared to other nations.

Susan, I applaud you for your dedication. Keep up the good fight.

Hi Anthony,

Thank you very much for writing this blog. A few summers ago I read a book about merit pay for teachers and a system for it in Denver. I forgot the name of the book but I remember that Dave Eggers was a co-author. In Denver, merit pay is not based on test scores. Instead, it is based on teacher professional development, attendance, working at a hard to staff school, and collaboration with other teachers. I think these are the kinds of things Arne Duncan should be looking at. Now if I could only remember the name of that book...


Nancy King, NBCT, Berkeley

Thanks for your comment, Nancy. I believe the book you are referring to is entitled "Teachers Have it easy; The Big Sacrifices and small Salaries of America's Teachers."


The Denver program to which you refer is called ProComp, and it was the result of careful negotiations between the District and the teachers' union. Leaders there went to the voters to secure additional funding for their innovative approach. See here: http://denverprocomp.dpsk12.org/

This is a critical issue which needs to stay in the forefront of our discussions as we move forward with a system that is being forced onto States/Districts through the awarding of "Race to the top" funds.

I do agree that the most common model of awarding pay raises based on graduate hours does not assure that we are becoming better teachers.

This District I have taught in for the past 17 years has had a system based on PD, committee work, leadership roles, etc. However, beginning next year we are adding a "student growth" component to our professional appraisal system. This is a form of merit pay. There is a test identified for each subject area which will be used to identify student growth. Our principal tried to tell us that it is better to teach lower students because it is easier to help them make progress. Hmmmm!

Our District did include teachers and administrators
in the development of this plan, but they were told that they must come up with a merit pay model.

Remember that all teaching staff are on the pay scale; social workers, speech therapists, Art and PE teachers, resource teachers, librarians... The system has to somehow fairly include and evaluate all staff based on student performance.

I think that overall, these systems will narrow instruction because teachers will focus on the skills and content being tested. It is not good for education overall.

I am disappointed in Obama and Duncan's shortsightedness on this issue.

I have seen research that states that students who have been held back are harder to move than those who are on grade level. My own experience supports this. Students performing at a lower level may have more "room to grow," but they are behind because they have not grown as fast as their peers. So it could be safely predicted that they will be harder to move ahead. And if they are concentrated in certain schools, and within schools in certain teachers' classrooms, you have a recipe for unfairness.

I teach in a Montessori program (public school) placed in a Title school. We are a very collaborative staff, many of us volunteering to teach before and after school remedial classes. Several of us also departmentalize depending on our talents and background (math endorsed teacher teaches math, teacher with a masters in reading recovery teaches reading, etc.). This would all be lost with pay being based on test scores. Teachers would no longer want to have students from other rooms in their remedial classes before and after school, and teachers would stop sharing students. We would stop sharing what works for each of us. Obviously we can't all get raises, so it becomes each teacher for him/herself. I would no longer be able to take time to teach math to the gifted students in the building, because I'd be too concerned about how my own students were going to be doing on the test.

I even dislike the term "Race to the Top". A race implies winners and losers. We are so much a "winner take all" society, that I don't want it to sneak in and destroy our classrooms and students, too.

I teach a group of gifted students - may are in the top 1st and 2nd percentile for their age. Most people would consider this a dream job - and I do love it - but if I'm going to be evaluated on the improvement of my students, who are already achieving the highest possible score on standardized tests, how will I show growth? My students' growth looks different; I don't try to take third graders from an 8th to a 9th grade reading level, but rather, expand their knowledge and literary experiences. It's difficult to meausre gifted students against typical standards.

I have taught Exceptional Students for 16 years. Presently we are producing passive, non-risk takers for the 21st century and this hit or miss method of merit pay is simply too narrow. We have lost our focus. Our country needs a major paradigm shift. The present method of accountability is much too simplistic, for society and individuals are complicated and deserve a more respectful, more wholistic method.

Thank you for the thoughtful comments. Debbie makes a very important point sometimes overlooked by those who wish to motivate us by using rewards tied to outcomes. One of the greatest values held by teachers is our desire to collaborate and support one another. This is also one of the hallmarks of an effective school. When teachers have a strong voice in designing alternative pay systems, as in Minneapolis, there is an emphasis on working together to improve instruction.

I work in a district with a great many beginning teachers. Our experienced teachers are stretched thin trying to provide support and mentoring for them all. How would this be affected by a program that increases your pay only for the growth of your own students?

Found it. The name of the book is Teachers Have It Easy: The Big Sacrifices and Small Salaries of America's Teachers. It was co-written with Daniel Moulthrop and Nineve Clements Caligari, and published in 2005.

One problem that I have with this discussion is that it tends to look at "merit pay based on test scores," on the one hand, or continuing with the current system of non-evaluation on the other. The second problem is that the discussion is heavily peppered with an assumption that the goal is to pay 'em more or get rid of 'em.

Modern systems of evaluation--based on research in management and social science--exist for the purpose of improving the quality of work output. A key assumption is that an employee needs outside feedback in order to understand the needs of his/her employer, see his/her work as others see it and to set mutually agreeable goals for improvement. These things are not terribly far removed from the things that research in education reveals are supportive to student learning.

It is not unhelpful to have teachers involved in developing evaluation systems--so long as their motive for involvement is not to subvert the process by insisting that any method of evaluation is unfair, that we all know already how hard teachers work and why don't they just leave us all alone. We have to move away from the teacher as widget (all the same) point of view and adopt an assumption that every valued employee, no matter their longevity, has room for improvement.

This involves building trust (a two-way street) between teachers, administrators and other stakeholders in the process of education.

If the goal of merit pay is to improve teaching perhaps we would be better served by paying teachers more, to attract a bigger better pool of applicants, and then making it easier to get rid of the bad teachers. At most every school everyone knows who the really bad teachers are and usually nothing gets done about it. Get rid of those who don't do their jobs and replace them with people who will.

Walt--I think that casting the wide net is pretty much what we do now. We assume that those who leave the field in the first three years are the ones who shouldn't be there. Some make the case that other countries do their selection through more thoughtful means, allowing far fewer to enter into teacher programs to begin with (and some require a masters before entering the classroom)--however they may expect that there is a job for them at the other end--and no outstanding school loans. Holding teacher education resources constant, if we made such a switch in this country, the quality of teacher education might also improve.

I work in an urban district with a high turnover rate. Our evaluation system is limited by the fact that it falls entirely on the administrators' shoulders, and often they are overwhelmed by other more urgent (though not necessarily more important) issues. I would like to see teachers take a stronger role in providing feedback to one another, building a sense of what good teaching looks like in their school together.

But Walt raises a good point in terms of the base pay. Right now in California there is a vast difference between the salaries paid in one district compared to another. And often, these differences follow socioeconomic lines. So teachers in Palo Alto top out at over $100,000 a year, while in Oakland, the top is below $70,000, and likely to fall further soon. Plus it is tougher to teach in Oakland. So we lose a great many teachers after they have been here only two or three years. If a teacher chooses to stay in Oakland for their entire career they could be giving up potentially hundreds of thousands of dollars they could have made if they went elsewhere. Teacher base pay must be adequate, and it should be made more equitable, otherwise we continue to have a system that favors the wealthy in every way.

I read the book "Teachers have it easy," and think there is a strong parallel in medicine with Primary Care Physicians. If both teachers and Primary Care Physicians were paid according to the value they create for society, the world would be a very different place. Both fields operate at a fundamental level.

Comments are now closed for this post.


Most Viewed On Teacher



Recent Comments