eduwonkette_header_515.jpg

Through the lens of social science, eduwonkette takes a serious, if sometimes irreverent, look at some of the most contentious education policy debates. (Find eduwonkette's complete archives prior to Jan. 6, 2008 here.)

« And the Winner is.... | Main | Watch Out, Elizabeth Green, Erin Einhorn, & Jenny Medina!!! »

Who Slipped a Mickey in John Merrow's Kool-Aid?

Kool-AidMan.jpg
It wasn't me, that's for sure. John Merrow shed crocodile tears this morning over public education's "upside-down universe where student outcomes are not allowed to be connected to teaching." He gives a special shout out to New York, though conveniently fails to mention that our tests are administered smack in the middle of the year. We'll give him a pass for forgetfulness - but watch your drink next time, J.

You can't swing a fish anymore without hitting a glitch in value-added models - try some of the papers from the recent Wisconsin Value-Added Conference on the complexities of measurement error (value-added models must contend with measurement error in both last year and this year's scores), interval scaling (few tests are scaled so that 1 unit of growth at the bottom of the scale means the same thing as 1 unit of growth at the top of the scale), and non-random assignment (see Jesse Rothstein's new paper on just how large these biases can be). Or you can refer to these earlier posts:

* skoolboy on: The Status of the Status Quo in Education Policy
* More Signs of the Apocalypse! (More on NY's Teacher Tenure Law)
* After NY's Teacher Tenure Law, Blogosphere Plays Union Pinata
* My Value-Added Bucket List
* Do Value-Added Models Add Value? A New Paper Says Not Yet
* The Oops Factor in Measuring Teacher Effectiveness
* Ignoring the Great Sorting Machine
* No Teacher is an Island
* What Does It Mean for a Teacher to Be Good?

Alexander Russo, also commenting on Merrow, makes the mistake of equating teachers' evaluation of students with tests and quizzes with the evaluation of teachers by students' test scores. It's just a bad comparison. Teachers give tests, assignments, reports, homeworks, etc in order to evaluate students and to see what they've learned. These measures are part of an extended interactive process through which a teacher hopes to move students forward. The purpose is not simply to label a student as "good" or "bad" based on one assessment. But when we evaluate teachers based on students' scores, the teacher is being evaluated on a more narrow set of skills than are students, and high-stakes are attached to a single test. So the intent of the process is different; few value-added plans are designed to help teachers improve, but focus instead on assigning rewards and sanctions.

The measurement issues are also different. In an elementary school year, a teacher probably collects 900 data points on student performance (let's say 5 a day); with teacher value-added, we end up with 20-25 data points a year. Teacher value-added is, in short, a low precision enterprise. Readers, what do you think of Alexander's comparison?

Happy weekend, everyone!

TrackBack

TrackBack URL for this entry:
http://blogs.edweek.org/cgi-bin/mt-tb.cgi/3925.

Comments

It's not clear to me that teachers are absolutely opposed to all assessments. And, even if they were, I'm not sure that the fact they assess students all day would make them bad people.

I'm saddened that John Merrow got sloppy in thinking through the difference between "student performance" and "student performance data."

He wrote accurately that "Now, administrators are required to 'drill down' ... to pinpoint who seems to be an effective teacher." But he didn't even speculate about what percentage of administrators have met that requirement.

Merrow said that "I have met with superintendants, principals, and department chairs armed with this data" ... who identified teachers' strengths and weaknesses, presumably in a fair manner.

How many superintendents, principals, and department chairs has he met who haven't had that data or ability or fairness?

Merrow correctly wrote that "Forward-looking union officials would push for creative uses of student performance data – such as using it to help teachers in areas where the data reveals they are not reaching their students."

But no union official can move forward without evidence that data would be used in a fair and accurate manner.

I don't know the specific personalities, and perhaps Merrow is not as convinced that the proposals coming out of NYC are as fundamentally untrustworthy. I suspect Merrow is making a political point with this article and I suspect he'll try to keep his ideas even-handed, keeping the process moving forward. After all, if he was my administrator, I'd have no qualms being evaluated by whatever data he chose.

Merrow is fundamentally on our side, and I expect he'll push the other side to show some integrity. Until then we should remember Merrow's description of NCLB and his own experiences seeing schools that have reverted to "drill, drill drill"

Given the NCLB experience, I would think that Merrow and others could be convinced that the first step is to kill dishonest accountability schemes. Then we need to move together to create real data-driven accountability.

i think alexander is making a powerful, nuanced point :-)

despite differences of intent and measurement, there is a fundamental, underlying similarity between teachers evaluating students and teachers being evaluated that should not be ignored -- if only to help teachers understand why it is so frustrating to some non-educators to hear teachers do what seems like resisting the common-sense notions of evaluation.

plus which, let's not idealize what's in teachers' gradebooks and why it's there. not all testing done by teachers is all that formative and constructive. many teachers sort and label kids with no real intent or ability to help students improve, just like they don't want to be sorted and labeled by others.


Alexander,

What would you cite as evidence for your claim that "many teachers sort and label kids with no real intent or ability to help students improve"?

I was just about to write what Skoolboy wrote. I find that remark stereotypical and offensive.

It's not really constructive to say that many people from one ethnic group are criminals. It's not much of an improvement to slur members of a profession.

Alexander, I'll help you with this. You can use the two "teachers" whose rooms are on either side of my classroom, a number of others who teach elsewhere in the building, or any countless others around the city.

It's funny that the defense of teachers - the begging for proof - always comes from those who spend every moment making sure that nobody can collect that proof. We can't use data until they're both perfect data and perfectly-and-fairly-used, according to the fans of the status quo, because that would be unfair. Are we to believe, then, that the 900 assessments that elementary school teachers perform each year are all 100% fair and providing 100% solid, useful data? If not, why would we subject our kids to anything we wouldn't subject ourselves to? The answer, I'm afraid, is that the people who argue against testing are either solely or mostly concerned with how whatever system we employ benefits the adults, rather than their students.

thanks, everyone --

for more discussion on this, check out the responses to my original post about teachers and testing:

http://scholasticadministrator.typepad.com/thisweekineducation/2008/05/student-tests-a.html#comments

The Russo post exemplifies how far out of touch our most vocal education policy commentators are -- does he really think that student progress and outcomes play no role in the current system of evaluation for teachers? Does he really think that noone had ever thought of looking at classroom outcomes before Fordham, Ed Sector ad nauseum starting blabbing 24/7 about it?

Socrates,

There is no question that many or most gradebooks are full (or are supposed to be full) of primitive assessments. It is equally true that students have property rights, and the imperfections in the grading systems would face dubious futures in court battles. And it is equally true that these primitive situations are common throughout education. Is any of this news?

The question is whether we want to blame others or create a better system. Why is it that administrators in high poverty neighborhood schools are completely unable to provide backing for teachers who try to raise academic and/or behavioral standards? Administrators have to assume that if a student's family tries to assert their rights, the school won't be able to defend its actions. (which explains why some in management want to push ahead with such dubious evalution systems. If they have to "prove" with a degree of confidence that a teacher is ineffective, they just have one more task on an impossible "To Do List.")

So you want address this situation by taking away the property rights of teachers? Would you gamble your career on how you will be evaluated by a bogus system? We need to recruit and retain more talent, not drive it away.

Being a 55 year old and a classroom veteran, the lack of crediblity that pervades the system does not necessarily effect me. If I need grades to motivate, except in occasional situations, I conclude that I have failed. I don't need disciplinary backing to do my job within the current situation. (But I will honestly admit that I can't get my students to the point where they will pass graduation exams unless the system changes.) The problem is young teachers and teachers who are attempting to teach their students to read for comprehension and to achieve real numeracy. Our best can get our students to progress greatly from the 5th and 6th grade skills that they bring with them to high school. Some of the best rely on grades.

But we can't get our kids close to the level they need to be(to have a healthy and hopeful future) unless we reform a full range of dysfunctional systems within our educational systems. One of which is creating a fair and reliable method of efficently removing ineffective teachers.

Anyone can find numerous proiblems within our system, so assessing how bad things are is no great feat. The question is how do we make things better. I simply can't understand how you can make things better for students, especially those who have been treated so unfairly by family, economics, and history, by just attacking teachers.

By the way, how much time have you spent in schools in spring? Have you experienced to full court press which teachers endure as administrators pressure us to ignore our academic and attendance standards? How would you respond to the nonstop guilt-trip we get to ignore behavior that were never be tolerated in any other institution?

If I want to play a constructive role in our district, I can't get all judgemental on administrators who are struggling with these dilemmas. I wish outsiders would bring a similar understanding to teachers.

It would be much too embarrassing to not correct one thing I said. Being a high school social studies teacher, I try to help teaching literacy and numeracy, in addition to appropriate behavior and my subjects' Standards. Real world, though, I teach literacy and numeracy compensation skills.

And real world, I'd welcome any support in making the system better so I don't have to always use the preface "real world."

John,

I agree with just about everything you wrote about the classroom. I, too, consider it a failure if I have to hold grades over the head of my students to get them to perform, and I too teach literacy and numeracy even in the content subjects, though at a much lower grade level than you. As for whether I've been in a school in the spring time: yes, I have, for more than 10 springs, as a matter of fact.

I disagree, though, that we can't be judgmental toward our administrators. I hate it when our administration insists that we promote kids who have skipped school, started fights, failed classes, etc., as long as they've passed the tests. I consider them either to be committing a crime, or being complicit in its commission. I do sit in judgment of them - it's not OK to hurt kids in this way.

At the same time, though, if we want to hold them accountable, we have to give them the tools of accountability by which they can manage their teachers.

Socrates,

I sure don't want to monopolize the discussion, but I have to apologize about the phrasing I used. I should have remembered that you are a teacher. Can I blame my mother for making me rush on Mother's day?

How about this? We need to make judgements about the way central offices have distorted policies, but we shouldn't get judgemental?

And since outsiders love to big-foot it into schools from the top down, lets pass this mandate. Before issuing the final draft of any accountability proposal, its authors have to reread Catch 22.

I agree that we teachers need to be held accountable for our work and that administrators need better tools for determining teacher accountability. My colleagues over at Teacher Leader Network and I have been advocating for some time now that an effective teacher accountability system requires, in part, more rigorous and comprehensive assessment of student learning than what most schools or district now use (walk-by, oops, walk-through evaluations or standardized test scores alone).

Does anyone know the status of the Performance Standards Consortium in NY? They were working with a model of comprehensive, multiple measures of student performance that got them exempted from the state's Regen exams. I'm in another state, but could it be a model of the types of data about student learning upon which accurate and fair teacher evaluations could be constructed?

Post a comment

Ground Rules for Posting
We encourage lively debate, but please, no profanity or personal attacks. By commenting, you are agreeing to abide by our user agreement.

eduwonkette

eduwonkette
E-mail me

The opinions expressed in eduwonkette are strictly those of the author and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Get RSS

Get eduwonkette delivered by e-mail. Enter your e-mail here:

Delivered by FeedBurner

Advertisement
Powered by
Movable Type 3.34

EW Archive