Get instant email alerts from EdWeek's blogs. Learn more.

« A Thought That Might Help Explain Public Denial | Main | When Tests Don't Measure Well What They Appear to be Measuring »

What's Wrong With Merit Pay

| 56 Comments

Dear Deborah,

Over time we have developed a very solid and smart community of readers who like to argue with us and with each other. That is as it should be. And of course we need to bridge differences—or disagree—with them, too, as we do with each other.

So the subject today is merit pay. This is an important topic because it has become clear that President Obama has decided to hang his hat on this idea. It has not yet been explained just what he means by merit pay. Does he mean that teachers should be paid more for teaching in what is euphemistically called “hard-to-staff” schools? Or paid more for teaching in areas where there are shortages, like certain kinds of special education or subjects such as math and science? Or paid more for mentoring other teachers? Or paid more for teaching longer days?

I would call such compensation “performance pay,” rather than “merit pay,” because teachers are paid more for doing more.

But I have a feeling that what the Obama administration has in mind is paying teachers more based on their students’ “value-added” test scores. So if their students see increases in their scores, they will get “merit pay” to reward their supposedly superior teaching.

I believe that this is the direction the administration is heading and that it is the purpose of the millions that will be spent on data warehouses in every state. And it is why Secretary Duncan has told the governors that they will get their stimulus money only if they collect and report data to the U.S. This was an odd request because some of the data he asked for is already available, such as the gap between state and NAEP scores (previously published in Education Week, for example, and no secret).

There are several reasons why it is a bad idea to pay teachers extra for raising student test scores:

  • First, it will create an incentive for teachers to teach only what is on the tests of reading and math. This will narrow the curriculum to only the subjects tested.
  • Second, it will encourage not only teaching to the test, but gaming the system (by such mechanisms as excluding low-performing students) and outright cheating.
  • Third, it ignores a wealth of studies that show that student test scores are subject to statistical errors, measurement errors, and random errors, and that the “noise” in these scores is multiplied when used to make high-stakes personnel decisions.
  • Fourth, it ignores the fact that most teachers in a school are not eligible for “merit” bonuses, only those who teach reading and math and only those for whom scores can be obtained in a previous year.
  • It ignores the fact that many factors play a role in student test scores, including student ability, student motivation, family support (or lack thereof), the weather, distractions on testing day, etc.
  • It ignores the fact that tests must be given at the beginning and the end of the year, not mid-year as is now the practice in many states. Otherwise, which teacher gets "credit," and a bonus for score gains, the one who taught the student in the spring of the previous year or the one who taught her in the fall?

I believe that our readers are right when they predict that merit pay of the stupidest kind is coming. I predict that it will do nothing to improve our schools. A few weeks ago, the conservative Manhattan Institute released a study showing that merit pay had no impact on test scores in 200 schools in New York City that are trying it. In fact, scores went down in larger schools that offered bonuses. This little experiment in schoolwide bonuses is costing taxpayers $20 million a year.

Now it is possible that scores may go up in later years; this is only the first year, after all. But what is most interesting is the subdued release of this study. When the Manhattan Institute releases a study, it often holds a press conference to announce the results. This study, however, had no fanfare; its study was quietly posted on MI's Web site; no press conference, no press release. Somehow I suspect that the study would have been released with bells and whistles if the scores had flown upward.

Here is my prediction: Merit pay of the kind I have described will not make education better, even if scores go up next year or the year after. Instead, it will make education worse, not only because some of the "gains" will be based on cheating and gaming the system, but because they will be obtained by scanting attention to history, geography, civics, the arts, science, literature, foreign languages, and all the other studies that are needed to develop smarter individuals, better citizens, and people who are prepared for the knowledge-based economy of the 21st Century. Nor will it identify better teachers; instead, it will reward those who use their time for low-level test preparation.

Is it possible to have an education system that mis-educates students while raising their test scores? Yes, I think it is. We may soon prove it.

Diane

56 Comments

How about taking all the potential 'merit / performance pay' funds and instead pay extra to teachers who go into neighborhoods and teach through community-based projects?

Rather than guess at what kids 'know' via bubble-tests, we could see what they 'know' via active service and community involvement.

If you want kids to learn better, help make their communities more viable.

Dear Diane & Fellow Readers,

Great post. As a part-time history professor, my concern recently (bear with me) is that I'm seeing more and more students who don't understand the purpose and construction of footnotes. While that's admittedly a particular complaint, a significant portion of my time is also spent teaching students how to read and write in relation to history. The kind of literacy needed to understand history is increasingly not there. Lest I sound too cranky, this has been going on for as long as I have been teaching (just short of ten years). But I've noticed an uptick in the past few years, and fear a greater proportion of those with decreased historical literacy in the future if testing concerns in relation to math and science increase.

Yours,

Tim Lacy
Chicago, IL

Diane,

I think that you left out some other important issues.

Perhaps most important among is the lack of evidence that this year's highest performing teachers -- by this value-added measure -- are likely to be next year's. If they are not, then we are talking about something that is a product of luck and statistical noise, rather than of skill or expertise. It would be like paying bonuses for teachers whose student grew the most -- in terms of height! In other words, entirely beyong a teacher's control.

This gets to the issue of "instructional sensitivity," an idea that we should always mention when we talk about paying teachers for test scores. Quite simply, our tests are not designed to measure teaching quality, and we don't even know how to design such tests. If it is possible -- and it might be -- we would need years of research and the money to pay for it before we roll this stuff out.

But I wouldn't want a repeat of Reagan's strategic missle defense (aka star wars). Politicians promised wonderful things far in advance of the technology of the time, and billions were spent to develop such systems, without any useful product ever resulting. The dreams of politicans and outsiders can't make something possible just because they want it to be so.

And so, we are left with the underpants gnomes problem. People know what they want step 1 to be (in this case testing). And they know what they want the outcome to be (in this case, better teaching). But they are stuck with that middle part.

Perhaps it would be better to move past the assumption that testing kids is the appropriate first step, and instead work back from how to get better teaching.

Diane,

I have thought and read about merit pay extensively over the past couple of weeks. Your concerns/arguments against it are indeed compelling. You almost have me convinced it's a terrible idea.

However, before we pass judgment on it we'll need to hear specifics from Obama/Duncan as to how they plan to implement it into our schools. In addition, if we do not adopt value-added merit pay a viable alternative will need to be adopted to replace the fraudulent teacher evaluations we have in our schools now. They are a complete waste of time and resources and we cannot completely dismiss the use of merit pay until something more quantitative and less subjective is considered.

As well, the reauthorization of NCLB is going to have to significantly reduce the amount of testing in our schools for me to look upon the amended legislation favorably. Whether that includes random selection of students annually, less grade levels tested, or a combination of the two, somehow we need to test less. All things considered, that could prove to be quite challenging.

At my school, we have overcome a number of the problems you mentioned by giving a computerized test to all students early in the fall and again in the spring. Tests are given for reading. language arts, math, and science. The teacher's effect is determined by the amount of growth a child makes. Cheating is virtually impossible because the computer gives each child a completely individualized test that is guided by the answers the student gives to each question. And no one can "game" the system because everyone is tested. Overall I am very pleased with the feedback I get from this system. I do have some misgivings. The tests are time consuming, and they have to be given in addition to a number of state mandated tests. As computerized tests they don't call for the written responses that are so helpful in revealing deeper levels of thought. To a certain extent this is compensated for by the state test which does include essay questions.

I support objective measures of teacher effectiveness because subjective measures leave even the best teachers vulnerable. One of the most extreme examples of this is the appalling treatment of Jaime Escalante and the core of highly effective math teachers who worked with him. You probably know that Jaime Escalante led a high school calculus program that had some of the highest AP calculus pass rates in the country, and this in a district that served a largely poor Hispanic population. What you might not know is that an assistant principal once threatened him with dismissal for coming in too early and staying too late. He was removed from his math department chairmanship. After he and the team of high achieving math teachers he worked with were pressured out of the school, the AP calculus pass rate at Garfield dropped seven fold. I believe that most principals are more fair and even-handed than the one at Garfield, but it only takes one small minded administrator to ruin a teacher's career. Objective measures of teacher effectiveness such as value-added achievement tests can help to protect teachers from the problems inherent in subjective evaluations.

It is increasingly evident, at least to me, that the differences being bridged here are indeed becoming fewer and fewer -- for all of us.

That said, your points are all worthy and, I think, valid.

I think there is nothing inherently wrong with the idea of "merit pay". But before we go off making major new policy, wouldn't it be refreshing if we thought things through, got the thing right and addressed the substantive issues?

Recently I have been reading E.D. Hirsch, Jr.'s "The Knowledge Deficit", and while I don't agree with everything he says -- and I certainly have some issues with what strikes me as an overly structured curriculum sequencing -- the knowledge-content focus is, I think, a sound one.

To go along with a significant set of revisions to the nature and content of assessments, we need to address the knowledge-based content of what is being taught.

Then we could talk about merit pay.

Thanks for a coherent argument against what appears to be a done deal. People with computers and snazzy-but-opaque statistical models see every problem as an opportunity for data analysis. Sometimes, more data obscures viable solutions, and redirects the conversation in unhelpful ways.

It is, however, possible to track teacher impact on student learning without relying exclusively on standardized or computer-delivered tests. Every experienced teacher should be able to articulate what her students have learned, supported by believable evidence, connected to her stated learning goals. Some of this data may be quantitative, but across the spectrum of disciplines and developmental levels not all assessment data will be numbers. And "quantitative" (as you clearly demonstrated in your analysis of the problems with tests) is not a synonym for "objective."

In fact, I would argue that training teachers to identify and defend convincing evidence of their own students' learning would dramatically raise both instructional efficacy and student learning.

I also believe paying teachers for the performance indicators you listed, and including external evaluations of practice as well as demonstrated impact on student learning, would be much more productive than our current model. A system that rewards 30 years of staying in place and random credit accrual isn't moving us closer to our educational goals.

The problem with teacher created assessments is that they are subject to wishful thinking. This problem would surely be exacerbated if such assessments were to become the basis for teacher evaluations.

Consider what happened in Nebraska when they tried to use teacher created STARS assessments. According to the STARS tests the achievement gap had been nearly eliminated and 75% of African American students were reading on grade level, but NAEP tests showed that only 9% of African American students were actually reading on grade level. You might argue that the NAEP tests were wrong, but if the achievement gap was really are narrow as the STARS tests indicated, it's difficult to understand why only 39% of Nebraska's African American students were graduating on time as opposed to 83% of white students.

I am not trying to imply any kind of bad faith on the part of these teachers. I work in a high poverty, mostly minority school. When I first started teaching, I was sure that my students were doing perfectly fine. It took the results of the state tests to open my eyes to the problem. It's natural for a teacher to assume that what is typical of the students at her school is "on grade level".

In addition, it is hard to see how simply letting teachers argue that their students are performing at high levels would offer any protection for teachers. Would Jaime Escalante's principal have treated him any better if Mr. Escalante had tried to show his effectiveness through samples of student work? I have a dismal picture in my mind of Garfield's principal dismissing his school's seven-fold drop in the number of students passing the AP calculus test by arguing that "everybody knows" that standardized tests don't measure anything of value. Subjective measures of teacher effectiveness give administrators too much power.

Hi All... hope this finds you well.

Diane and Deb i want to thank both of you for creating this site... it is an excellent model of dialog and depth that is sorely missing in many places!

Diane... i think you have hit another key point in your last comments.....

"Is it possible to have an education system that mis-educates students while raising their test scores? Yes, I think it is. We may soon prove it." (Diane)

This really extends the entire conversation outside the realm of "urban school reform" into what will
put all our schools on a path that leads to mis-education at a time when we could be transforming our educational systems.

We seem to all agree.... schools need to transform....

but as ackoff says: doing the wrong thing right...is much worse than doing the right thing wrong.

I believe we are on the road of continuing to do the wrong things right!


be well... mike

Hi,

As a special educator, merit pay worries me. I know my students can learn and make gains. But if you are a new young college bound student who actually wants to go into education today... Why on earth would they choose special education.

We have IEP's which get longer and more legal every year. Then we have the behavior plans and alternate assesments (which in Ohio at least measure little more than how well I can guess what the readers are looking for). And now, your chances of getting merit pay if you go into special education instead of math--probably significantly less.

Teachers need to be held accountable, and all students can learn. But I don't think we have a good way of measuring effective teaching. Which leaves us giving merit pay based on unreliable, unvalidated test scores.

I took a class period to teach my students with some fine motor deficits how to correctly bubble in tests. And then I go home at night and wonder how that could be considered quality education.

Thanks again,Diane, for a very concise summary of the issues. Alfie Kohn coined the summary phrase: raising the scores and ruining the schools.

Pay for student scores has been tried before, and not worked. George Madaus has written on it, among others, and tells me has a chapter on it in a new book just released by IAP (I am traveling and don't have more info here).

Current payment structures may be flawed, but the envisioned system is really rooted in what has worked so very well on Wall St recently - payments based on a narrow set of criteria that can be gained. We have recently see how well that works over any extended period of time. Rather than having teachers compete for what likely will end up declining sums, anyone interested in teacher compensation should get to work on fair forms of payment.

There is no reason that a reasonable process of teacher evaluation (largely nonexistent in US schools by most accounts) needs to be tied to payments, but can be used in developing teacher and school quality. Indeed, helpful feedback tied to means of improvement should be normal operating procedure - and have nothing to do with standardized test scores.

What Nebraska did not have and other systems have done is what is called moderation: take samples of work (portfolios if you will, or a set of tasks/proejcts) and re-score them. This can be informative feedback or lead to re-setting grades. In either case, it addresses the concern that has been raised about Nebraska and, I believe, about objectivity (tho probably would not solve the problem of particularly bad administrators, but then giving such people more power, as many current 'reforms' call for, would exacerbate it.) BTW, one of the goals of the NE approach was to improve the quality of teaching thru such things as teacher involvement in ground-up reform, teachers learning to assess their students better and thus better understand each student as a learner. Chris Gallagher's book describes this well. But NE never got past the early stages. Oh, yes, they did not necessarily need NAEP for the gaps to be seen - every district had to use an NRT at three grade levels.

If one finds major gaps in learning outcomes, the real question then emerges: what to do. Unless the nation is willing to address the impact of poverty, then supposed solutions might in some cases lead to better schools and more real learning (something very much worth doing) but many of the demonstrable gaps will remain.

Thanks again for the sharp post and for fostering interesting discussions.

What are the objectives merit pay is supposed to meet?

Better scores on reading and math tests?
How does this benefit society? Does this translate into a better prepared workforce? Will we all be better capitalists?


Attract more quality teachers? Or discourage anyone from entering the profession unless they are Math or ELA teachers? I'm sorry, Mr. Holland, but your students' scores on the standardized marching band exam were too low this year; I'm afraid we'll have to let you go...

Diane,

Thank you for pointing out some of the confusion regarding "merit pay." People define it in many ways; often "merit" seems tied to longer hours and years, as though the truly dedicated and qualified teacher would automatically put in the extra official hours (I am not talking about unofficial hours).

Such confusion of language is an example of what Demiashkevich called the "witchery of words." We can see such "witchery of words" in the very groundwork our districts are laying for merit pay.

Almost everything is in place for merit pay right now. The "Quality Review" rubric now specifies that teachers and students must set "learning goals" regularly. What are these "learning goals"? Well, they can be many things, but lately they have been taking the form of numbers and letters. Students receive a reading score and set a goal to attain a higher score. Students reach a "reading level" (a letter from A to X, according to the flawed Fountas and Pinnell system) and set a new letter for themselves.

Teachers are already receiving data on their students' "progress" in scores and reading levels. "Inquiry teams" are examining individual students' progress (as determined by scores) and identifying methods and solutions to be applied to the schools as a whole. (For instance: if a certain software product is bringing up scores, then more students should be using it, the teams conclude.) All of this is a step away from overtly judging teachers by their students' test scores.

Students are affected just as severely as teachers. In conferences students must state their "goals" in terms of scores and reading levels. They must know those scores and reading levels by heart. They must know their short-term and long-term goals. It matters little whether they have read anything of value, so long as they are making "progress" on the computer and with the Fountas and Pinnell levels. It matters little whether they have read a poem that stayed in their minds, a story that delighted them, a book that they found very difficult but kept on reading anyway because it was so interesting. In fact, they are discouraged from reading highly challenging books; the "just-right" book (again, determined by Fountas and Pinnell level) is deemed more appropriate than the book that fascinates them but is a bit over their heads.

I see children yearning for challenge. But challenge does not reflect immediately on scores. Challenge can create the appearance of a setback while the student struggles with the material. Schools would rather play it safe.

Certainly we need to look at students' progress. Certainly tests, including standardized tests, play a role in this. The danger occurs when the test results are taken as ultimate truth, when the scores become more important than what the students actually learn.

Diana Senechal

Thought this passage, from the creator of the TV show THE WIRE -- David Simon -- speaking with Bill Moyers -- mighte be relevant here:

DAVID SIMON: You show me anything that depicts institutional progress
in America, school test scores, crime stats, arrest reports, arrest stats,
anything that a politician can run on, anything that somebody can get a
promotion on. And as soon as you invent that statistical category, 50 people
in that institution will be at work trying to figure out a way to make it
look as if progress is actually occurring when actually no progress is. And
this comes down to Wall Street. I mean, our entire economic structure fell
behind the idea that these mortgage-based securities were actually valuable.
And they had absolutely no value. They were toxic. And yet, they were being
traded and being hurled about, because somebody could make some short-term
profit. In the same way that a police commissioner or a deputy commissioner
can get promoted, and a major can become a colonel, and an assistant school
superintendent can become a school superintendent, if they make it look like
the kids are learning, and that they're solving crime.

Diane, thank you for the clarity and urgency of this important post, I hope it is widely read and seriously considered. I too feel desperately worried about the administration's rush to embrace merit pay and the "value added" bandwagon everyone seems to be jumping on. What seems to be terribly lost in the debate is a moral point - we can't just do things to increase the effectiveness of teaching, we must also ensure that the means to desired ends are morally defensible ones. Virginia Richardson, in a speech at the University of Michigan in 2007, said it very well, in the context of NCATE beginning to push for K-12 student outcome data as a means of evaluating teacher education programs:
"So why argue against the use of student achievement scores as the outcome measure of interest? Probably because success on getting students to learn curriculum material—as measured by standardized tests is, by itself, a simplistic, inadequate, misleading and perhaps immoral measure of teacher education quality. We must also look at the ways in which teachers conduct their classrooms, and at other elements that affect the success of teaching including such aspects such as willingness and effort by the learner, a social surround supportive of teaching and learning, and opportunity to teach and learn."
Another moral implication is what happens to the sometimes fragile community inside schools when money is used to favor some and disregard others. It's hard enough to develop the crucial professional learning communities that foster mentoring, support, and formal and informal learning opportunities for educators. Why must we mess it up by throwing bonuses at those who make the test scores go up?

While you make many valid points about the problems with using standardized tests as the basis for merit pay, the comments above suggest you've taken too narrow a view of how merit or performance pay might operate. It is imperative that we identify and encourage teacher growth as well as student growth. The question is how to do so in a fair way that provides incentives for continuous improvement. I agree that many standardized tests as currently designed and implemented are not adequate. However, computerized adaptive assessments do offer more nuanced information about student progress on specific learning standards. What would be helpful if there was consensus about standards for teaching excellence that would guide our discussion about evaluating teacher performance. Certainly we have to look at student growth to assess teacher effectiveness, but many other factors could also be evaluated, including parent satisfaction, contributions to the overall school, collaboration with other teachers, organization and planning, etc. My only fear is that by focusing on these inputs we again lose sight of the ultimate goal: student achievement. We might also think about merit pay as a reward at the school level, rather than for individual teachers, to foster professional learning communities and peer support and pressure for excellence. We might also think about merit pay not merely as extra cash, but opportunities for leadership, professional development, dissemination of effective practices, etc.

Dear Diane, Good stuff!

I would begin here by naming the topic what other knowledge worker-professionals call it. Pay. Or, Compensation. Just as a nod to equality with other professions.

So, how did we get into this pay at piece-rate bs? And what will follow when it inevitably turns to disaster?

It started with the demand by teachers that they all be paid the same, based on seniority. Not on how the work is done, as most every other professional is paid, but merely on time spent in place.

As individuals, most teachers recognize the unfairness of this, both to themselves and to students:
- It means that only the very worst teachers are ever worked out of the system. The merely deeply mediocre ones live on to pick up a very nice retirement package, frustrating their students, parents, and of course the decent teachers who must witness day in and day out the cost of learning that follows.
- It insures that many people who would make great teachers are deterred from entering the profession.
- It makes it difficult to switch careers to teaching.
- It makes it difficult to grow the payroll, as boards confront one huge step 3-5 year increase, rather than the gradual stepping up of wages based on individual cases.
In short, education continues to embrace a compensation model that does not work in any other profession.

The question begs: ‘why hasn’t this all broken down long ago?’

Part of the answer I touched on yesterday. Teachers who make it a 30 year career have found a nice way to make the capitalist system work for them. If you enter at 24 and leave at 54 it’s a nice living overall. Teaching being a relatively satisfying profession, most are content to let the SEA leaders and organizers insist that the system not be messed with.

Yet, as you point out, there is much pressure to change the system now. The most politically acceptable way is to pay based on test scores. Moderately informed people know that kids in many schools are not learning, when kids at schools nearby are, and they want the gap bridged. Workers also know that time-in-place pay scales didn't produce good results in their own workplaces. Thus, pay for test results is the political reality that has come to the fore. For that, teachers have no one to blame but their leaders.

Probably everyone here agrees that basing pay strictly on test scores will come to awful results. The question is, how widely will it be implemented, and what will follow on?

Ideally, this type of merit pay would be widely implemented, fail, and be replaced by something which made sense. Something which allows supervisors to judge the quality of the work being done. This is what works everywhere else, its absurd to think it won't work for teachers.

There is, of course, an intermediate path. This is the one used by the federal government. There, a schedule lays out the levels of service. In theory its based strictly on years experience and years education; in reality if your boss sees you as performing at a higher level, you'll be hired in at a higher grade. Or quickly promoted thereto.

---
Tim above reminds us of the frustration college professors are having with students of all walks. It didn't start recently with testing, and isn't just the urban dropouts that are an issue. There's this vast ongoing middle area, too. Plus there's the shortage of STEM graduates entering the workforce.

The 'good' news is I doubt any 'merit pay' will soon become law. The President will propose it. Maybe some districts such as NYC will attempt it. But the current Congress in no way will let any strict measures force such merit pay on the states. We need look no further than the removal of the 1700 vouchers to see how beholden they are to the whims of the NEA.

So, relax, all, merit pay is not coming to you soon. Students, ...sorry.

My link to the US GS pay schedules was blocked above by the blog software.

For a number of years I have had an idea for merit pay for teachers. I described it once in a comment to some blog and it was called "dreadful", and a few other bad things. But I persist.

Call it a popularity contest by secret ballot, but here's my plan. Everyone gets to vote for their favorite teacher, or their favorite three teachers. The students get to vote and their collective vote counts for 20% of the total, with proportional payout. Parents get another 20% of the vote. Teachers get another 20%. (Each teacher can vote for himself or herself and two others.) The principal also gets 20% of the vote, and the superintendent gets the last 20%. An accounting firm is hired to conduct the vote and tabulate the results. The available money is divided up according to the results as determined by the accountants. Every ballot is secret, and there is no appeal once the accountants have spoken. Everyone can argue endlessly about the process and the results, if they wish, or they can just shut up live with imperfect democracy (or whatever we might like to call it).

My feeling is that a system like this, for all the criticisms it would invite, would be a workable system. Money would be spread around with at least some connection to merit. Is there a better system?

Diane and Deborah -

Perhaps one of you should consider a response to Thomas Friedman's NYT OpEd titled "Swimming Without a Suit"
http://www.nytimes.com/2009/04/22/opinion/22friedman.html?_r=1&ref=opinion

His heart may be in the right place, but I am tired of him writing without really knowing what he's talking about.

To borrow from Warren Buffett, "Beware of geeks bearing statistical formulas." You would think that Diane's laundry list of "issues" with basing teacher pay on students' standardized test scores should be enough to kill the notion. Further it's inconsistent with all the literature regarding reinforcement. The Bush administration foisted off the statistically untenable "Adequate Yearly Progress." To date, the Obama administration is piling on even more foolhardy notions.

Student instructional accomplishments do not line up neatly with grade by subject matter categories that can be tapped by kids filling in bubbles on multiple choice test items.

Moreover, some students will learn with no instruction and some will learn despite mis-intruction.

Moreover+, the school principal, (the "instructional leader") and all the other "leaders" at district, state, and federal levels are held harmless for their actions and remain unaccountable.

After taking into account the issues Diane raises and these additional considerations, the implications are (1) we need transparent indicators of instructional accomplishments, and (2) "merit" is a function of the team of personnel at the school level, not at the individual teacher level.

I've sketched one game plan that operationally responds to the requirements:
http://ssrn.com/author=1199505

It seems to me the game plan beats the popularity system Brian proposes, although his notion would be one reasonable way to go about dividing up the merit pay pool.

I'm sure there are other ways to reasonably implement "merit pay." But those currently being incentivized by the "reformers" are toxic and doomed to failure. This time around, we should be clear that the respsonibility lies with the "reformers," not the "reformees"

Ooops. Pardon the mis-spelling and missing period in the last sentence. I meant to hit "preview" by clicked prematurely on "post."

Brian,

I agree with you on many things, but this popularity vote is not one. Some of my favorite teachers at various times were not especially popular. It would have been heartbreaking to see them lose a contest like that.

Of course there are good teachers who are popular and not-so-good teachers who are unpopular. But popularity and quality are often at odds, especially in the moment. My U.S. history teacher was not universally liked, but I adored her. She told me it was not enough to love ideas; I had to learn the facts.

One of the dilemmas of education is that we do need to see short-term results, yet much of learning shows itself over time and in subtle ways. Both of these are true; we cannot ignore one in favor of the other. But a reasonable combination of the short and long term, of the visible and invisible, is a matter of human judgment. It cannot be reached by formula.

Some may say the formula is fairer than a capricious principal. Not so; it is simply mechanized caprice.

Diana Senechal

Brian, Diana, the system described is essentially what happens at most workplaces where knowledge-workers and skilled workers practice: each year the raise pool is determined and allocated by management. Management doesn't work in a vacuum.

Each day in a group of 100-200, thousands of conversations go on. Much of these conversations will go to how each team member is contributing. If you're generally helpful, the word will spread. If you lie often enough, you'll be exposed. If your work is competent but off target, you may be out of favor this year, but a genius next year.

Either way, it is all of the contributors Brian listed (customers, fellow employees, supervisor, her boss, who determine your share of the pie. In any given year it may make no sense whatsoever. Over time, though, it does. And, in the odd cases where five years go by and you are not recognized as the insightful kind, warm, person you are, well, someone down the road will see it clearer.

Its not the impartiality of a system with hard lines, cold numbers, and no room for human judgment. But it gets us new drugs, quality electronics, safe autos, reliable planes, etc.

Merit pay if it only refers to paying teachers related to test scores is obviously problematic. Equally problematic is the fact that teachers who teach in suburban wealthy districts get reverse “merit pay” when they are paid more, have better benefits and working conditions for teaching students who have the most assets and are by definition easier to teach than teachers in poor districts. There has been a federal program for decades to reward teachers for working on Indian reservations, why not the same for inner city math teachers. Maybe we should re frame "Merit Pay" and call it "Work incentives for difficult environments". Are you opposed to paying teachers more to work in districts that will never have enough money to pay as much as the suburbs, or are geographically undesirable (inner city or remote reservation)?

"it gets us new drugs, quality electronics, safe autos, reliable planes, etc."

Ah. That's the big difference with el hi. These outcomes are all concrete. And there is a basis for judging the contribution made by individuals. Certainly it's a judgment call by management, based on all the input sources mentioned. There is always some grumbling, but by and large, those involved consider it "fair," and there is typically some adjudication mechanism available to those who feel mistreated.

There is no counterpart to this in el-hi. Certainly not before before the categorization of "courses"--and even there it's a very fuzzy matter.

The school principal is an "administrator," not a manager, and has little or no information about what goes on behind classroom doors. Standardized achievement tests are insensitive to differences in instruction, and "proficiency" is defined in terms of arbitrarily-set cut scores on ungrounded statistical scales.

All instructional failures are attributed to student "deficits."

Bottom line. There is no basis for assigning "merit."

In other sectors "experience/seniority" does add value and typically added responsibility. That's not how the history of el-hi instruction has evolved to date.

The administered-but-not-managed personnel structure is anachronistic but it's not going to be fixed by railing at "unions," by establishing charter schools (that are unionizing), or by the palliative of "value added" schemes.

El-hi personnel certification and school accreditation are controlled by the unaccountables in "higher ed." "Reformers" might think of turning their accountability armament in a different direction.

There is something about this discussion that disturbs me because it seems to go zooming past a crucial point. First off is the great leap forward from data warehouses to the assumption that merit pay is coming. But, that is not what is troubling me the most. In all of the conversations that have to with teacher evaluation, this issue of pay gets rolled in and then things start to get all weird. Not surprising. This is generally the way things go when money is involved. That is how we (either as humans, or Americans) seem to be structured.

I would prefer that we take a step back and think about what education is for--the mission, the vision, the big picture. Somewhere up there is something that we can agree upon having to do with education in the grand scheme of things having to do with learning. The point of any evaluative process within the education system ought to be about how learning can be maximized. This isn't unique or new (or even very "reformy").

Teachers, as all others within the system are entitled to an accurate system of evaluation and feedback on their work. There are myriad ways to do this--most don't exist in schools today, or in many other workplaces either. But any organization that does not have a regular system of evaluation of both employees and processes will ultimately suffer for it.

Those of us who have been adults for a few decades or longer have probably experienced some of the good, bad and ugly when it comes to evaluation--as well as its complete absence at times. To be effective, evaluation must be supported by identifying and providing the means to improvement, as well as having the ability to separate individual from systemic weaknesses.

Certainly the ability to link student scores to individual teachers is an important part of cultivating that ability. Developing a cadre of administrators with requisite management skills and knowledge is also important. Further research into particular qualities that exemplify successful teachers is also pretty high on the list. Brian's idea actually has a distant cousin in 360 evaluations--which involve the contributions of select individuals who provide multiple points of view regarding the work of the individual.

Personally, as frustrated as I am with the sense of entitlement that goes along with guaranteed employment as long as no one is murdered or raped, I would hold to keep evaluations far away from pay issues. It just doesn't help. There is too little trust in the system as it is. Certainly it is worth boosting the pay, or offering other things of value (fee parking, tuition, low interest home loans) to teachers who are worth keeping in "hard to staff" schools, or content areas, as well as those who take on leadership roles (and would it be too much to pay the debate coach on a par with the football coach?). But, I think the "merit pay" issue is really just serving to muddy the waters of another issue far more important, which is how can we use evaluative measure to support teacher learning and growth (and that of students)?

While Diane's objections are undeniably valid, we as educators must not lose sight of the fact that the existing system of teacher compensation flies in the face of any form of meritocracy. Our free market economy, valued the world over (until recently), is based on rewarding people willing to work hard(er) and do a good job.

For teachers to be paid based on the number of years in service plus graduate credits earned says little to nothing about their actual classroom performance. Can they relate to kids, motivate them, challenge their curiosities, meet their individual needs, encourage them to do more, nurture them, get them excited about learning, etc. etc?

Our existing system has proven itself to be totally ineffective. Worse, it has fostered mediocrity, evidenced by Tom Friedman’s piece in today’s New York Times. It’s a dumb system and beyond that, it’s wrong regardless of the reservations of anyone. THE EXISTING SYSTEM MUST BE RADICALLY AMENDED. It must be changed and the change must include reasonableness and fairness. Teachers who do a better job should be paid more. Teachers not performing well should be offered guidance/mentoring towards the remediation of their inadequacies and helped to improve. If they are unwilling to work to improve they should be encouraged to find another profession.

It has allowed too many mediocre and unproductive teachers to spend a career in the classroom. While this may sound benign on the surface it is exacerbated exponentially by the students who have suffered from having these teachers.

And Diane, the onus is now on you. You’ve voiced your concerns over value-added merit pay not being the answer so now it’s up to you to come up with a viable alternative. Come on. No procrastinating will be tolerated. Chop, chop! NCLB will be up for reauthorization within the next six months and Obama will incorporate his ideas unless a more pragmatic option is presented. If we can be of any assistance just sing out.

Margo/Mom,

I don't think the discussion has gone "zooming" past that central point at all. Diane's column and a number of the comments address "the mission, vision, the big picture," as you put it. But the "big picture" is definitely missing from current proposals for merit pay tied to test scores.

It is not simply that things get weird when money comes into the picture. It is that test scores are dominating school culture now. Teachers are supposed to sit down with their students and say, "This is your reading score, and this corresponds with such-and-such a reading level. Now, what is your goal for June?" The student is then supposed to state a goal. Then they break it down into short-term goals (again, test scores and reading levels).

This is so strange to me, it seems I grew up in a different world. I grew up believing that the "level" of a book didn't matter nearly as much as its contents. All through my childhood I was reading very easy books, very difficult books, and books in between. I return to Winnie-the-Pooh today, for my own enjoyment.

Why have we divided books into these artificial Fountas and Pinnell levels? None of the books I read as a child would have fit one of these levels. Now books are being written to match the levels exactly. If you have seen a book that has been written for one of these levels, you will know what I mean. It is fake stuff.

Yes, students do advance in reading. Yes, a third grader should be reading more advanced material than a second grader. But it is a mistake to define the levels and gradations with too much precision or to assume that children cannot make leaps. It is just as big a mistake to focus on whether a book is level M or level N, rather than on beautiful and challenging passages in the book.

What does this have to do with merit pay? Everything. These Fountas and Pinnell levels are part of an attempt to make a "science" of Balanced Literacy. They are a melding of constructivist practice with "data-driven" instruction. It is a business alliance. If the Balanced Literacy program has a built-in way of measuring "progress," then it is guaranteed millions of dollars for years to come. And teachers will be judged (merit pay or not) on their adherence to the BL program and their "success" in moving their students up these meaningless levels.

Yes, we need to look at the vision, the big picture. We need to teach literature, not reading levels.

Diana Senechal

"First off is the great leap forward from data warehouses to the assumption that merit pay is coming."

There is $200++ million already out to all but 4 states to develop "state longitudinal data systems." There are grants to states for implement various "value added" test results, some of which are tied to teacher pay. Secretary Duncan introduced "merit pay" in Chicago and has spoken positively about the notion in connection with the "better teachers" priority.

So at least "small leaps" have already been taken. On the other hand, the State of California state legislature has specifically legislated that teachers not be specifically linked to student test scores--students and teacher information is in separate data bases, and teacher interests are very likely to similarly act--for the reasons Diane has listed.

But we don't need to worry about "sending in the clowns." They're already here.

To respond to John Stallcup's question. Los Angeles, several other CA districts, (and my understanding that districts elsewhere) have been offering "bonus pay" for some time to teach in "challenging locations" and in some subjects like math and science. This practice has been accepted without much to do. But it's a very different matter than the "merit pay" practices that Diane is referring to.

Diana:

(sigh) I guess I grew up on a different planet. We didn't have Fountas and Pinnell, but we had SRA Reading laboratory--which progressed neatly through levels. It wasn't terribly interesting, but it was the latest thing and once you were finished you could generally do some "free reading" if there was time. We also had three reading groups for our (earlier) Alice and Jerry experiences. We all knew who was ahead of whom. We also had a big chart on the side wall displaying the column of stars for each student as they turned in their book reports (mine was never one of the top one. I liked reading--book reports weren't so much fun). As readers, we all compared notes: number of pages, difficulty. We knew.

Apparently I am still living on another planet. In over a decade of sitting through IEP meetings I can scarcely recall a teacher who was able to articulate a measureable goal and break it into achievable benchmarks. I still remember the principal who told me it would be illegal to use an actual reading level as an annual goal. Personally I would have been delirious if any teacher that my kid had was setting reading goals and helping my kid to track them--I don't care who made money on putting out the reading system.

I think that we need to go back a few steps to before there were scores (or at least scores that were comparable and public). The best we could do with regard to accountability for kids learning was to count the books and the dollars available in each school (and even that got to be pretty difficult at the building level). Maybe show some concern about what came out the buildings at the end of the day--how many graduated, how many went to college, how many won scholarships.

It's pretty hard to make a case that the data-less school (or the data-hidden school) was turning out the best and most equitably educated students possible. We don't have to be concerned only about the test scores. How about including graduation rates? How about considering discipline rates? How about looking at discrepancies between kids with and without disabilities with regard to either of those rates? Can we shift some concern to attendance? What about incidences of violence? What about opportunities for listening to parent concerns? Every single one of these data points has a connection to student learning. In fact, attention to these things has a very good likelihood of improving test scores (and learning).

Looking at end of year achievement test scores to the exclusion of everything else is a choice. It hasn't taken the place of looking at all other elements that might go into evaluation and planning for improvement. The problem is that evaluation and planning for improvement was never a part of the picture to begin with (systemically speaking). This doesn't mean that teachers were not free to evaluate and improve on their own (individually). But in the absence of evaluative data at a system level, there cannot be any systemic improvement.

Something that seems to get overlooked by proponents of merit pay: lots of teachers are currently working their tails off and STILL getting crappy results. It seems an underlying assumption of the merit pay advocates is that the lure of extra pay would get teachers to try harder. But many (maybe most) of us are already trying very hard WITHOUT the lure of merit pay. Many of us (myself included) would not work at all harder for an extra 5 or 10K (a balanced life is too important). A much bigger problem than unmotivated teachers, IMHO, is teachers without sound ideas about teaching. If E.D. Hirsch were as well-known and understood as John Dewey, our teaching force would be much more effective that it is now --using the same amount of raw effort.

Touche, Ben. The same people who are fond of saying, "You can't fix schools by throwing money at them" are contending that you can fix them by paying some teachers a pittance more on the basis of test results that are sensitive to SES differences, not to instructional differences.

It's not the teachers. It's not the kids. It's not the water. It's the texts and tests. The texts and tests go unexamined, while while kids are given inadvertent toxic instruction and tested with insensitive tests. It's the instruction, stupid.

The wish and whip, "standards and standardized tests" policy is a failed policy. Test scores have remained essentially flat and the IES evaluation of Reading First provides evidence that the expenditure of $6 billion over 5 years had no impact.

The beltway consensus is delusionally headed for wishing harder with national standards and whipping harder with data warehouses. President Obama is well-intentioned, but he's getting very faulty educational intelligence from the framers of the failed policy.

Dick and Ben and Margo and Diana you all bring good points,...all pointing to the problems of bureaucratization. The tests, the lack of literature, the second graders who cannot read at any level, the 70% dropout rates...all signs of bureaucracies strangling out the human desire to learn.

So, ...anti-bureaucratization? We're all agreed this is the solution?

Because I think this is known as smart injections of entrepreneurialism. Seriously, how else do you reduce bureaucracy?

No, Ed. Think some more, and look at the evidence.

Privatization only brings greater crime and corruption. Even the best private schools don't do THAT much better than public schools. The public school enterprise is capably administered. That's a strength, not a weakness. The glitch is that the enterprise is completely unmanaged in that there is no reliable feedback on the consequences of the instruction being delivered that can provide a foundation for improvement.

The federal government is attempting to micromanage school sites through flow down mandates to the states. Bus 101 should provide enough "knowledge" to dismiss that practice.

All the ills that are easy to note do not originate at the school site level. They are all instigated by the unaccountables at the top of the EdChain.

Mitchell Hirsch,

You are right about the Tom Friedman article. It shows him at his worst. He was writing about matters he knows nothing about. Like Nicholas Kristof, he is just blowing smoke, maybe he just needed to toss a column off quickly, and who knows more about schooling than McKinsey & co?
Diane

Dick:

I think that you are right in pointing up the distinction between administration and management--as well as the lack of meaningful and timely feedback (which I would hope that a responsible evaluation system would have as a goal).

Senge points out the results of consequences systemically separated from actions (although I am not so sure that he uses that particular terminology). If we think for example about middle school being a secondary "customer" of elementary school (the student, of course being the primary customer)--and the lack of communication between those two entities about the "product" (learning) being sent from one to the other--perhaps we might begin to pick up clues as to what kinds of systemic feedback might be helpful. I believe it is Deming who talks about separating out common causes of flaws (NONE, or most, of the 6th graders arrive with an understanding of the causes of the Civil War--or whatever)--that require systemic reform (including Civil War in the curriculum, or re-ordering the sequence, or improving the feedback to the teachers teaching it) and the specific causes (Mr. Jones' students tend to lag behind in reading, repeatedly, over time) that require specific cures (Mr. Jones' hates to teach reading but turns out terrific science students--so maybe he swaps kids for the teaching of those subjects). Deming, of course, as a statistician deals with far more complex indicators than those that I throw out--but I think that the point is still valid.

But--whatever weaknesses in the current testing system--and I suspect that in reality they are somewhat fewer and milder than many folks would have us believe--it does provide a source of data, which while it may not be wholly on target all the time, may certainly be put to good use, when combined with other possible sources of data--both quantitative and qualitative.

Student testing data alone may not be sensitive to instruction--however when combined with meaningful classroom observation, may begin to be instructive. The end result of teaching (that is learning) is by nature far removed from the realm of the classroom. Students are likely to be years down the road before the consequences of teaching are fully experienced.

Using data for management purposes, so far as I can tell, is not a part of either the education or experience that we require or value from school administrators. In fact, we frequently cannot get past the single most important requirement of paying dues in a classroom (and then deride administrators for having "left" the classroom--either due to supposed incompetence or blinding ambition). As Deming recounts the "Japanese Miracle" it began with educating managers at upper levels in what he refers to as "simple but powerful" statistical methods. Seems like a good place to start.

"Using data for management purposes, so far as I can tell, is not a part of either the education or experience that we require or value from school administrators."

It would be hard to find a school administrator who would not place a high value on "data" and claim that instructional decisions are "data driven."

One of Russell Ackoff's "Management F Laws" is:
"An ounce of information is worth a pound of data"

The only "data" standardized tests yield is a number on a statistical scale. That's not informative. Item analysis information would yield more useful data, but the items are confidential.

The test data rank students, but they give no feedback to anyone re instructional programs or provide any clues regarding optimal next instructional steps for a student.

All of the so-called merit pay systems I have seen discussed by educators are sure to do more harm than good. And please, don't equate paying more for what is really a different job (more hours, tougher venue to teach in, etc.) as having anything to do with merit pay. It may come as a shock but in the real world, different jobs justify different pay scales.

I worked for Hewlett Packard for over a decade and experienced the best merit pay system I have seen. It was flexible, covered the person's total performance, and it was imminently fair. Of course, since it is not an "inside education" construct it could never be considered in the NIH (not invented here) education environment.

However, it could not be used in education because the education leadership is ubiquitously poor. They haven't been trained in even the fundamentals of human psychology and certainly not in the best research proven management techniques. They only know how to preserve the status quo.

It seems that sometime in the future when we are so uncompetitive globally that we must face reality we might be able to do some things that should have been done decades ago. But that will mean doing something that educators have been singularly unwilling to do. That is, every option that might improve performance by requiring a little sweat and pain on the part of educators is rejected without being seriously considered. That is the primary reason things get steadily worse over time in the education fiefdom; defensive, delusional, insular and inbred.

With respect to the current merit pay discussion, doing nothing is preferable to doing the wrong thing.

Dick:

I don't equate "data" with "test scores." These may or may not provide the appropriate information at any given time. However the fact that the latest buzz-word (data-driven) has come to mean single minded concern with test scores only illustrates my point--and that made by Paul Richardson. There are in fact models operating within the US that put to use organizational/management theory, are able to discriminate between kinds and sources of data and select the ones most appropriate to determining the impact of proposed changes, and determine which systemic changes will likely bring about needed changes in outcomes.

Within education, we don't put a whole lot of value on these things--they are either invisible (and therefore impossible) or part of some evil "business model" intended to do us all in. The idea that "non-educators" could make a contribution is scorned and seen as dangerous.

Looking back to Deb's suggestion of denial, test scores--in forms more public and comparable than ever before--have utility on the scale of determining policy. While many claim that it is no surprise that there are ongoing (and sometimes widening) gaps in the learning of various groups, perhaps they would be surprised then at the ability of building level employees to diminish and explain away the existence of such gaps. Doing "pretty well" or "as well as can be expected," or even worse "as well as they deserve," is an infectious disease of denial that requires an ongoing reality check at a comparative level.

Does that tell anyone what to do differently tomorrow? No, it does not, nor should it be expected to. But that is where we have only picked up the mantle of being "data-driven." We need manager/administrators who are skilled at determining root causes, who understand how to develop and test a hypothesis--and in the end remove the onerous burden of improvement from the backs of individual teachers working in isolation from one another. Such manager/administrators should be able to present a concrete rationale for any decisions that expand what is working well or diminish efforts that have ceased to be effective.

If you pay attention to the "draconian sanctions" sections of NCLB, these kinds of steps are actually spelled out in detail. Strategic improvement planning is called for early on. In the roughly 5-7 years in which such planning progresses, and provides the bulk of "sanctions," (and while an entire crop of elementary students has progressed from kindergarten to middle school), there should be an ability to latch onto something that will bring diminish the number of students not proficient by the requisite 10% to "make AYP."

Of course, if the level of dysfunction is so high, or the knowledge regarding management is so low, that nothing improves, perhaps it is better to regard the school as a "new start" and try to reform under different leadership. This is a fairly logical response to deeply ingrained patterns of mistrust or baggage from earlier failures or simple lethargy expressed as "nothing works."

The fact that so little happens, in many schools, in those crucial 5-7 years, suggests to me that moving "leadership" up from the classroom to management positions--while touted as required in order to have people who "really understand," is a poor way of training and selecting leadership. Even McHamburger has better management training than this.

Imagine a hospital that demanded that all administrative posts be filled by former doctors and nurses who could "really understand." Imagine if they touted themselves as being data driven because they put "laser like focus" on morbidity and mortality data. Imagine if they rejected any health care research as coming from "ivory towers" where "no one understands what we deal with."

What if we just took Deming's observations in Japan and applied them here. What if we started training every high school principal and superintendent in "simple but powerful" statistical methods? Not a full major or minor in statistics and probability--but enough to be able to sort out what is or is not statistically significant, be be able to identify indicators close enough to a process to determine whether anything was happening--and the direction of change. Maybe we could turn around some of the blind grabs at solutions in search of a problem (like school uniforms), or make progress against some things that are really dragging at teacher morale--like parent communication or attendance or discipline, or the amount of time spent in documentation. Maybe we could have middle schools empowered to point out discreet problems to elementary schools and they could work together to see that kids needs are appropriate met across the transition. Does this mean paying teachers for test score? Not in my opinion. But--does it increase the likelihood of changing some dismal conditions indicated by the test scores? Yes, I think it does. Would that diminish some of the rabble who are crying for merit pay? Well--it certainly couldn't hurt.

Your position re "turnaround schools is confirmed by a recent study:

http://www.epaa.info/ojs/index.php/epaa/article/view/300/34

"Improving Low-Performing Schools through External Assistance:
Lessons from Chicago and California"

"This article describes the design and implementation of external support to lowperforming schools using data from Chicago and California. . . The findings suggest that the model of assistance employed in both Chicago and California was inadequate to the task. While the policies examined demonstrate recognition that low-performing schools need additional capacity if they are to substantially improve student outcomes, external support providers used limited and haphazard approaches, and as a result, the support component had little influence on teaching and learning."

It's going to take something other than training in "simple but powerful" stat methods.

What Secretary Duncan did not know how to do in Chicago, he now has a priority for the nation's el-hi enterprise to do as part of the ed stim package. Go figure.

I think the findings in Michigan were that it required a specific threshold of substantial reforms to arrive at actual "turn-around," but that this did in fact occur in some cases that exhibited success.

No--study of statistics alone is insufficient. But it might be a good beginning while we reform the programs that train administrators, and the criteria by which they are hired(or we could hire in already experienced people from other sectors--but there's a whole lot of resistance to that one).

As a high school principal at a charter high school in Colorado, I live in the center of the bullseye for merit pay. The discussion here is excellent, particularly the distinction between extra work, extra responsibility, market scarcity and merit pay. They are all different, and schools have to think strategically about what they need to reward. At my school, a perfect AP Physics teacher is more valuable than a perfect standard freshman teacher. They may be identical in performance, but one adds more value to our school. I can quantify and qualify that value in a number of ways without using a single test score. I can also differentiate between two standard history teachers without using a test score. Discriminating between levels of performance is not impossible, it takes complex evaluation and evidence-based standards. It is not impossible. But teachers unions and mediocre teachers—who are unwilling to submit to comparisons with colleagues—fight fiercely against meritocracy. Why do they fight so hard?

As a high school principal at a charter high school in Colorado, I live in the center of the bullseye for merit pay. The discussion here is excellent, particularly the distinction between extra work, extra responsibility, market scarcity and merit pay. They are all different, and schools have to think strategically about what they need to reward. At my school, a perfect AP Physics teacher is more valuable than a perfect standard freshman teacher. They may be identical in performance, but one adds more value to our school. I can quantify and qualify that value in a number of ways without using a single test score. I can also differentiate between two standard history teachers without using a test score. Discriminating between levels of performance is not impossible, it takes complex evaluation and evidence-based standards. It is not impossible. But teachers unions and mediocre teachers—who are unwilling to submit to comparisons with colleagues—fight fiercely against meritocracy. Why do they fight so hard?

Peter,

It's obvious why teacher unions fight so hard. If merit pay were instituted the need for unions would diminish significantly. There would still be collective bargaining and negotiations regarding contract language but the big issue - money - would essentially be off the table.

As for mediocre teachers - they're scared to death for obvious reasons. They don't want to be compared to superior colleagues for fear their compensation would either stagnate and/or their administrator could negatively evaluate them based on their tests results.

_________________________________________

Dick,

You state, "Item analysis information would yield more useful data, but the items are confidential." Here in Massachusetts the MCAS tests have provided item analysis for teachers and administrators to detect instructional deficiencies for years. Teachers have been able to scrutinize tests results to determine where they have been deficient and yes, it helps quite a bit.

Peter,

Your post shows clearly why teacher evaluations should not be left to the subjective opinions of administrators. Ask any teacher who has taught both AP classes and freshmen classes, and they will inform you that it's much easier to teach an attentive group of college-bound students who face taking an AP test than it is to teach a group of freshmen many of whom may be struggling academically. The AP teacher adds more value to your school only if you don't value the struggling students as much as you do the high performers. And testing data is especially relevant in AP courses. Would you really suggest that the seven-fold drop in pass rates for the AP calculus test at Garfield after Jaime Escalante was pushed out is of no concern?

I also take issue with the suggestion made by some posters that test scores are insensitive to instructional differences. The reality is that some are and some aren't. Clearly, the one-shot end of the year state tests will tend to reflect SES differences rather than instructional quality. But when students are tested at the beginning and end of the year, and the focus is on the improvement the students have made over the course of that year, the problem is less serious.

Merit pay is a problem for many reasons, but I see no reason why test scores shouldn't be part of a teacher's evaluation, if only to protect teachers from the possibility of a negative judgment by a capricious administrator.

Sorry Ray, since I identified myself as a principal it might not have occurred to you that I know something about teaching. I've taught the AP students as well as the struggling freshmen in three states. I hire, fire, supervise and coach both kinds of teachers. You only characterize my opinion as subjective because you disagree with it.

Since I work with teachers on both kind of assignments, I know the distinction well. Both take preparation and effort, but the AP teacher has demonstrated a level of content knowledge and pedagogical skill that is superior to the standard freshman English teacher. At my high school, my AP calculus teacher (he teaches both Calc AB and Calc BC) also teaches the lowest level of freshman algebra—those "struggling students" you mention. But because we are a college preparatory school, his unique ability to take students to the collegiate level has more value in the marketplace. I have several math teachers who do an excellent job with low and standard-ability students. However, only my Calc teacher has the demonstrated ability to take top students over the top. His number are Escalante-like, with 100% of last year's class earning a 5.

You have zero evidence to support your blanket assertion that administrative evaluation constitutes a subjective opinion. It is only capricious because you don't agree with the conclusion.

Your suggestion that administrators can't discriminate accurately is also baseless. I have dozens of teachers who submit to my evaluation and keep coming back year after year. We pay an average of 10% below our host district, so I know we're not holding them with money. Many of my teachers are former college professors and refugees from the traditional K-12 system, so they know something about instructional ability. Do you really disrespect their judgement so much to suggest that they would keep working under conditions of capricious, subjective, opinion-based compensation?

Perhaps it is hard to define mediocrity, but I know excellence when I see it—and I know how to reward it.

Public schools won't educate. In fact, they can't educate. Last, they shouldn't be in the business of educating because they are government entities - which means that it is inevitable that those schools will first and foremost educate, not for the benefit of the people forced to attend them, but for the benefit of the employees who work within them.

All of this talk of measuring public school performance will always yield infinite debate over the legitimacy of those measurements. Further, all attempts to assure that they are legitimiate or credibly requires eternal vigilance on the part of those who believe they have the ability to render such a judgment. Left out are the parents and students, who are the best judges of what is best for them.

It is time to pronounce public education a failure. Those that do well within the system do not do well because of public education but in spite of it. Their success has infinitely more to do with their personal characteristics (initiative, drive, ambition, work ethic, etc) than the characteristics of the schools. Further, if they have good teachers, it is because those teachers have gravitated towards the better schools, which are better simply because they have better students. All of this is done in defiance ofthe intent and rules and regulations of the education system. In other words, getting a good education is really a subversive act - a subversive act by those who possess incredibly traditional characterisitics that once were taken for granted as necessary to get educated.

The real problem with all of this is the welfare state itself: it has simply absolved individuals from any kind of accountability for their actions. Public education is simply training to be a ward of the state.

Dick, Paul,

We, too, have interim tests with item analysis in New York, but they are problematic. First of all, each question supposedly tests a separate skill. So if the student gets a question wrong, we are supposed to conclude that the student needs help with the associated skill. This is not necessarily so. An error on a "main idea" question does not necessarily indicate problems with "main idea." The student may have misunderstood the passage or question or thought about them in a way that the test makers did not.

Many schools use software that provides students with instant feedback on their performance. (How ironic that teachers are told not to correct students explicitly, yet these software programs are lauded for doing just that.) The problem here is that students are stuck reading test passages and answering multiple-choice questions every day. They may enjoy it up to a point, but it is intellectually deadening. The "progress" matters more than the substance of what they are reading. And make no mistake: there is profit in it. The software companies push their products hard.

Diana Senechal

Peter,

After reading your clarifying post, I am much more inclined to agree with you. A teacher who can teach both struggling math students and get 100% 5's with his AP calculus students is a phenomenon.

But not all principals are like you. Remember what happened at Garfield. You sound like an excellent principal, but not principals have your level of expertise. Administrator judgments will always be part of the mix in teacher evaluations. I simply suggest that using value added test scores as a part of a teacher's evaluation provides a safeguard against the minority of principals who are inclined to be capricious.

Note to Margo/Mom: Peter's last post indicates the importance of having a principal with a solid teaching background. His time in the classroom gives him a real insight into the challenges involved and adds credibility to his leadership. It also sounds as if he knows something about statistics as well.

"But when students are tested at the beginning and end of the year, and the focus is on the improvement the students have made over the course of that year, the problem is less serious."

That "the problem is less serious" is problematic--for the reasons that Diane stated. Item Response Theory by which the tests are constructed inherently renders the them unfit for this purpose. What is required to measure teaching/learning success with any sensitivity is an indicator where performance piles up at the bottom of a distribution when the instruction begins begins and performance piles up at the top of the distribution when expertise has been reliably delivered.

Such indicators can be constructed, but it requires a psychometric perspective in the tradition of Binet, Gutmann, polling and text analysis methodology, and Business Intelligence.

Going down this path illuminates what the teacher has accomplished and this provides a sound basis for evaluating teachers. But as I've said, the accomplishments typically involve more than a single year/teacher at the elementary level. And at the high school level the orientation requires closer and different attention to the specification of "courses" and to "course sequences" than commonly prevails.

Dear Principal Hilts,

I am curious, does your charter school accept children whose parents DO NOT apply?
And
Are there any specific requirements for parent action(s) during the school year?

Merit Pay:

Problem 1: I certainly know why my union leadership is where they are... they earn hefty paychecks and find themselves in a position to perpetuate our "need" of them. However, if teachers (particularly those of us who have our hearts and minds in the right place) could TRUST the administrative arm of the public education system... maybe there would be more receptiveness. Frankly, I would prefer an end to the need for unions if I felt I could trust that my principal, who I like VERY MUCH, wouldn't throw me under the bus at the first sign of trouble.

Talk about twisted... the AFT and the NFL are BOTH members of the AFL-CIO. I do not hear very many pro football players complaining about the lack of pay for their kids' exceptional teachers.

Problem 2: The President is full of vague generalities about his "Pillars." Maybe teachers would be more willing to listen if there were some concrete details about HOW this would implemented. There must always be a starting point in order to bring both sides to the table. The President does not want to take too much of a risk. The teachers see the possibility of going from the frying pan into the fire if they jump on board too prematurely. How can Arne Duncan say he understands teaching? He certainly has yet to produce a quality "lesson plan" for these ideas. Who ever heard of publicly announcing your "plans" without actually having thought them through and without having presented the details along with the overall general outline? Our current President seems to be doing so frequently... and not just about education. "Vet" a member of his cabinet lately?

Problem 3: Removing poor "teachers." Hey.... I am all for it. Why? Because they make my goals for the kids more difficult to accomplish. However, most school districts (until this year) have faced teacher SHORTAGES. Principals are scrambling for warm bodies to cover classes. Until the COMPENSATION GAP is closed, the profession will not attract enough of the best. U.S. Dept. of Labor Statistics show the compensation gap is still at approx. $20K annually compared to people with equivalent professional training and experience in the private sector. Why exert oneself when there is no reasonable base from which to create incentive. Teachers with Masters degrees and 15 years experience cannot afford to buy a 3bdrm/2bath house here - even in the depressed market. The school board's solution... use existing undeveloped school board land to build "Teacher Ghettos.." Just what we all want... to live on top of where we work.

Problem 4: Parents insist on Social Promotion. THEIR egos being the biggest culprit. I say get rid of the current grade level system (K-12) and replace it with a set of skills which must be accomplished/acquired before moving to the next level. Schools in Kentucky are using it (at least at the elementary level) with some success.

Problem 5: Parents who are "too busy" to get invloved (more than paying their taxes and getting their children to the front door), but are NOT too busy to actually call their chiildren on the cell phone WHILE the children are in class. Go figure.

Problem 6: The transition from the parent supporting the teacher to the parent attacking the teacher on discipline matters. Its EASIER to give "A's" out like penny candy and to keep the kids "tame enough" to get through the day... to heck with learning. Why? Because the communities have decided that we should not be right. No child can be "average" anymore. Everyone is gifted or has a disability which requires absolution from accomplishing much of anything. The labeling is out of control in the effort to jockey for making children look good on paper. A parent will gladly sacrifice a teacher who has taught vs. a teacher who makes the GPA look golden. A handful of individuals working in a school cannot "hold the line" against an entire community if it loses its moral compass.

The important thing is to move to a "individual student growth" model for measuring achievement rather than stick with the current system which compares this year's third graders to last year's third graders...

you can assess a teacher's impact, if you look at specific students, rather than general aggregates....

Rory, we don't have a mechanism to enroll parents who don't request. This obviously means we have a population of parents who made a choice. This is the case in all choice systems, but many traditional public districts are now a hybrid of default + choice. BTW if you know of involuntary enrollment into a charter school I'd like to learn about that.

We require nothing of parents other than accepting the benefits and drawbacks of attending a charter school.

If teaching and administration were a purely voluntary association, then wouldn't both parties have a strong incentive to associate with the strongest partner? That's my world. We have no contracts, no tenure and no salary schedule. Every offer lives or dies on its mutual benefits. Every teacher, principal, and worker must deliver or depart. Every compensation offer must attract or it fails. We are surrounded by lots of charter schools, public traditional schools, online schools, and private schools. I can only attract students by attracting talent. I can only attract employee talent by providing a compelling work experience combined with acceptable compensation. So far so good.

Diane, I share your sentiments. Too much focus is on teacher performance. It's probably because of a system that permits, even fights for, very poor performing teachers to stay in the system, but I think they are in the minority, but it's one of those things where people judge the entire system by the few bad apples.

Money is misused and wasted in education on curriculum and tools that don't really improve education while lining the pockets of big business. I think we could improve education tremendously if we could create a standard curriculum for reading, writing, and math in the early grades that we know works for the largest number of students.

From my experience, I believe our students are performing worse on average today because teachers are forced to teach using ineffective curriculums and pedogogical tools.

We need to change the objectives. It's not to have every student perform at the same level. It should be to help every student grow to their individual potential. Some sadly will never perform to high standards.

Finally, let's change the qualifications for teachers. There are many highly qualified scientists, mathematicians, and writers with outside world experience that could add so much to education, and we make it difficult for them to enter the classroom.

We have it backwards. Teachers start their teaching careers at a young age after graduating college. I'd rather see experienced professionals enter the classroom in their later years when they have so much more wisdom and experience to share with the children.

Principal Hilts,

I do not know of a charter school with involuntary enrollment... that is why I was asking. I feel I might embrace the idea of Charter Schools more if such a school exists. It would likely be an opportunity for an "apples to apples" analysis.

Based on your description of the teacher/administrator relationship....
1. Who decides if "the goods" have been delivered?
2. Administrators likely earn less than teachers?
3. Do admiinstrators have the authority to tell a teacher to "depart"?
4. Does a teacher have the authority to tell you to "depart"?

I work within that "hybrid" you mention. I teach at a Magnet High School within an extremely large district (32 public high schools of whch two are the most populated in the nation). Participation in our program is voluntary and there is usually a lottery because we have too many applicants for the space.

We also have several charter schools - with mixed results. I struggle with the Ed. Secretary's (and the President's) claim that Charter Schools are creating a better environment and greater opportunity. It may very well be the case. However, they are not being forthright about the conditional variables. As in our Magnet School, the customers are quite different, inherently creating dynamic change from the start.

An aside....Nothing is as frustrating as a parent who, because of the chaotic environment in the neighborhood school, enrolls a child in our program thinking the proverbial "bullet has been dodged" and then, leaves all further participation, interaction, and responsibility at the front door.

Do you experience this in your school?

Those of us trying to persevere in the public schools... sometimes feel as though the public is being told there is a BETTER alternative to what we are trying to accomplish. If Charter Schools included ALL potential customers... well... the day to day conditions would revert to those which were supposed to change.

If an entire district wanted to try this, I would be ALL FOR IT. Otherwise the delivery of opportunity is not equal for all "stakeholders" within the district.

Comments are now closed for this post.

The opinions expressed in Bridging Differences are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Follow This Blog

Advertisement

Most Viewed on Education Week

Categories

Archives

Recent Comments

  • hertfordshire security installers: Greetings. Great content. Have you got an rss I could read more
  • http://blog.outsystems.com/aboutagility/2009/04/challenges-of-scoping-and-sizing-agile-projects.html: I would like to thank you for the efforts you've read more
  • http://acousticwood.net/mash/2008/03/yeah_off_to_the_uk.html: Between me and my husband we've owned more MP3 players read more
  • buy cheap metin2 yang: When you play the game, you really think you equipment read more
  • Nev: Anne Clark - If a Dr. instructs a patient that read more