Neither policymakers nor the public understands the complexity of estimating value-added models, so I preferred educating lawmakers and the public about what conditions would have to be in place to validly use these measures to nixing the use of test scores formally. Perhaps that was naive on my part, as Joel Klein wanted to ignore these limitations and move ahead with value-added (see his op-ed above).
But I worried that formally barring test scores from consideration would give union bashers another opportunity to distract attention from the larger problems faced by public education. And now the union pinata match is on. Joe Williams' post stands out for its histrionics. Featuring a mushroom cloud, Williams prognosticates, "When we are all standing at public education's funeral someday in the near future, remember to do a cough-chant of "murderer" when Dick Ianuzzi or anyone else from NYSUT tries eulogize." Kevin Carey digs deep and pulls out Paris Hilton-worthy dramatics: "It's hard to imagine a more unambiguous declaration of the union's total disregard for student learning when its members' jobs are at stake." Socrates calls the legislators "union-mouthpieces." Joel Klein, in his op-ed, even blames unions for the existence of achievement gaps:
Protecting grownups rather than making sure students can read and do math is how our country has gotten into the educational mess it's in today. It's the reason we have shameful racial achievement gaps separating our white and Asian students from our African-American and Latino students.
That's why there are no achievement gaps in North Carolina and Texas!
Yet none of these guys acknowledges the elephant in the room in New York: tests are given in January. That means that a value-added measure would estimate the effects of teacher pairs, not individual teachers: one teacher teaches students from January to June, and another from September to January. Even if two teachers are equally effective, a novice 4th grade teacher who receives students from a 10 year superstar 3rd grade teacher is going to look better than a novice 4th grade teacher who receives students from another novice teacher.
If NYC wants to get serious about value-added, tests need to be given in September and June, and these tests need to be designed to measure growth, which NY state's tests are not.
The good news is that principals are actually pretty good at identifying which teachers have high or low value-added, even in the absence of these data, and they can use this insight to inform their tenure decisions. Take a look at this paper by Brian Jacob and Lars Lefgren, based on a study in which the authors estimated value-added models, but also had principals conduct subjective performance evaluations. They found that principals can identify teachers with high and low value-added; for tenure, the goal is to deny tenure to teachers with especially low-value added. Moreover, Jacob and Lefgren found that, "a principal’s overall rating of a teacher is a substantially better predictor of future parent requests for that teacher than either the teacher’s experience, education and current compensation or the teacher’s value-added achievement measure." They concluded:
To the extent that the most important staffing decisions involve sanctioning incompetent teachers and/or rewarding the best teachers, a principal-based system may also produce achievement outcomes roughly comparable to a test-based accountability system. In addition, increasing a principal’s ability to sanction and reward teachers would likely improve educational outcomes valued by parents but not readily captured by standardized tests.
See below the fold for more wonky stuff on the testing calendar.
More on testing calendar issues:
Is there any way to accurately estimate teacher effects given the current testing calendar? If 3rd grade students are randomly assigned to the novice 4th grade teachers in the same school, teachers would get even shares of experienced and inexperienced teachers' former students, and they could be compared. Of course, we know that principals do not randomly assign students to teachers, so we have a problem.
But what if we wanted to compare teachers across schools, as Joel Klein would like to do? If teacher quality varies across schools (which it does), two identical 4th teachers - one at a school with experienced 3rd grade teachers, the other at a school with novice 3rd grade teachers (to simplify, I'm using experience to proxy teacher effects)- will have very different value-added estimates, and this is a problem that is not solvable by randomly assigning 3rd grade students to 4th grade teachers.
Is there any way to accurately estimate teacher effects given the current testing calendar? If 3rd grade students are randomly assigned to the novice 4th grade teachers in the same school, teachers would get even shares of experienced and inexperienced teachers' former students, and they could be compared. Of course, we know that principals do not randomly assign students to teachers, so we have a problem.
But what if we wanted to compare teachers across schools, as Joel Klein would like to do? If teacher quality varies across schools (which it does), two identical 4th teachers - one at a school with experienced 3rd grade teachers, the other at a school with novice 3rd grade teachers (to simplify, I'm using experience to proxy teacher effects)- will have very different value-added estimates, and this is a problem that is not solvable by randomly assigning 3rd grade students to 4th grade teachers.
I don't think that principals should be trusted with being the sole judge of teacher performance for tenure or merit decisions.
As a former teacher, I found that principals often favored (or disfavored) teachers based on factors that had little to do with classroom performance and teaching effectiveness.
It was often a matter of personality, or teaching philosophy, or whether your age or background matched the principal's.
None of which, I found, had much to do with how much students were learning. At the high school level, teachers were often retained for extracurricular duties, such as coaching (again, little to do with teaching effectiveness).
The problem of rating teachers is complex - but should not be left solely to the unreviewed judgment of the school principal.
We are told by the pundits and policymakers that teachers fear accountability. Maybe what teachers really fear is this: www.chron.com/disp/story.mpl/chronicle/5687425.html . Good thing NY has two years to study the publicly-available data and impact of the results.
Fear and Loathing, Never a dull moment in Houston!
DC - I don't think anyone's suggesting that we put this authority in one person's hands. Rather, my point was that principals/APs can see this thing we call "value-added" with their own eyes, at least for high and low performers.
On the testing calendar: My understanding of what NYC is doing is that a teacher's value-added for a given year is based on students from two different school years. Let's suppose that Ms. Scialfa taught 25 4th-graders last year, and 22 4th-graders this year. Both groups are tested last February and this February. Over this 12-month period, the 25 4th-graders were exposed to Ms. Scialfa from February to June, and to a new teacher from September to February. The test score gains for these students are apportioned to Ms. Scialfa and their new 5th-grade teacher according to the duration of their exposure to the teacher. So Ms. Scialfa gets credit for about half of the gains these 25 students made from last February to this February. Similarly, the 22 students this year were exposed to Ms. Scialfa from September to February, and to a different, 3rd-grade teacher from last February to June. Hence, Ms. Scialfa gets her proportional share of the gains that these 22 students made from last February to this February.
None of this, of course, addresses the learning consequences of summers, when school is not in session, and we know that poor and minority children are at risk of losing ground academically.
Would you be comfortable making high-stakes decisions about educators' careers based on a system that would make Rube Goldberg blush? I wouldn't.
Why not have teams evaluate teachers?
1) Their peers
2) Their supervisors
3) GASP Their students
4) Members of the community
5) Themselves (via a portfolio of everything they have done over the year)
Or is that too human for "us"?
That's much too sensible Philip.
Thanks for the details, skoolboy. It's bizarre to estimate teacher effects under the rationale that teachers vary widely in their effectiveness, and then turn around and simply assign a teacher effect (the time proportional allocation that you mention) that assumes the two teachers were equally effective.
There's another level of complication in NYC. They have pull-out and push-in teachers who teach some kids in a group and this is flexible. Schools also have literacy coaches - what is their impact or responsibility? And what of the class size issue? Any accounting for that? Or attendance? Say, there's a flu epidemic the week before the test. Or does the heat work or not work in the room? And what value added in relation to a few severe problem behavioral problems?
This law is not about whether test scores should or should not be used for tenure decisions.
The Union simply succeeded in making it more difficult for school adminstrators to dismiss new teachers before they gain the heavy protections of tenure. Any time the Legislature puts an evaluation factor off limits, you give the dismissed employee more ammunition to draw out a long legal battle for the job. Every fired probationer will now argue that the principal's decision was really based on test scores. I understand that tenured employees earn such heavy legal protection but non-tenured probationary employees -- like most of us in the working world -- should be subject to dismissal without cause. Yes, you may lose a few talented probationers but, on the whole, the system benefits from allowing
administrators to make non-challengable decisions with new employees and avoiding the expense and inefficiency of protracted litigation.
Eduwonkette,
With a few days' thought, I'm wondering how much of the ire directed at NYSUT is about the process of how this happened—i.e., without the "right" people in control or at the table. I suspect this is all about the waiting game going on with the end of the Bloomberg administration in New York. The use of value-added measures as the sole or primary tenure gateway is now blocked until after Bloomberg is out of office (and Klein is also likely to be gone). No one likes to be rolled politically, but the irony here is that many of those who disapprove of being rolled in Albany haven't said boo about others' being rolled in NYC.
SD: good point . . . very good point
Does anybody else get the sense that the arguments about this law are based more on emotion than reason? This post is probably the most rational one I've read.
AE: I object to the idea that new teachers should be subject to the same "at will" system as non-educators in the business world.
I've worked in both education and business settings. In the business world, supervisors have a profit and business motive to retain quality employees. In schools, there are no such outside influences on the principal- teacher relationship. No clients, no profits.
In consequence, arbitrary factors such as personal differences of opinion or personality conflicts are much more likely to come into play in the school setting than in a typical company.
In addition, in a business setting, employees generally have more than one supervisor (perhaps one direct supervisor and also a higher level boss). A termination decision is therefore more likely to be a joint consensus between more than one senior employee.
I therefore am more concerned about teachers' rights in an "at-will" world than I am of employees in a business setting.
DC Attorney:
I generally agree with your assessment of the conditions in education that make at-will employment problematic, but I disagree that this means we shouldn't have at-will contracts.
The problem is that principals don't have an appropriate incentive structure. Their personal paychecks could be made dependent on their school's performance in much the same way a businessperson's is.
The same goes for the two-supervisor structure you mention: principals are far from the highest employees on the DOE org chart, and there's nothing inherent about education that makes it impossible to have two tiers above each teacher (though really I would challenge the idea that more layers of bureaucracy mean more fairness).
The fact that at-will contracts might be problematic is a symptom of a greater problem, which is the lack of measurement and evaluation in teaching. If we had better measurement systems and used them to evaluate and compensate both principals and teachers, we could go to at-will contracts for everyone, which would mark a huge improvement for kids.
Socrates: I appreciate your comments. I agree with you that certain qualities of schools and teachers make at-will contracts more challenging to implement than in the private sector.
It's difficult to evaluate the performance of teachers with easy, objective metrics. If at-will contracts are to exist in education, I think more precise performance measurements are needed, as well as using input from more than one source (e.g., not just the principal) to evaluate our educators. I wouldn't necessarily use a supervisor higher up in the organization than the principal (as you suggested) b/c they rarely have direct experience meeting or observing teachers. My ideas would be more along the lines of using department heads, other teachers, or outside observers to confirm a principal's judgment before "heads roll" or tenure is denied.
I note that my perspective comes in part from my experience at one school where the principal wielded her tenure-recommending power to make first and second year teachers' lives really miserable ("Do what I say - or I will not allow you to stay in the school system."). I think any fair evaluation system should be set up to avoid abuses of power in which untenured teachers can be subject to the whims of autocratic principals.