« The Problem With Demanding Proof on Teacher Evaluation | Main | Boys, Girls, and NAEP Writing »

Value-Added is Not a Magical Black Box

We're now in day 5 of the Chicago teachers' strike. I found these quotes from the CTU attorney, Robert Bloch, interesting:

CTU Attorney Robert Bloch said Thursday the CTU finally felt on Wednesday that it was getting through to CPS negotiators on their concerns about the new teacher evaluation system.

"The system they are using to evaluate people is based on an extremely complicated, esoteric formula to measure student growth -- so complicated I think everybody on the CPS team will admit they don't understand it," Bloch said. "Experts developed it but not educators."

Gosh, and golly. Educators didn't develop indoor plumbing, air conditioning, and the internet, too--does that mean schools shouldn't use them either? More productively, I think an analogy to school buildings is useful here: When you're designing a new school building, you want to seriously take into account teachers' perspective on how the space should be laid out and used, what features need to be in classrooms, and so forth. But when it comes to, say, ensuring the correct structural support for the roof, you don't want teachers making those decisions. You want an architect or an engineer. You want an expert. Similarly, in designing the overall shape of teacher evaluation systems, teacher input (from all perspectives, not just the union) should play a critical role. But when you're designing the infrastructure to support that system, particularly the value-added measures, you want an expert. Unless, of course, you want the roof to fall in.

In addition, the complicated algorithms used to determine student growth -- called "value-added" -- are being debated nationwide.

"The problem is, how do you hold teachers accountable for improvement when so many things that are used to evaluate them are outside their control or very complicated?'' Bloch said.

"The science behind the student growth aspects of testing is untested and uncertain, and you're going to risk a teacher's career based on some guy in a back room writing algorithms or students who are not tested in the subject you're teaching?

"There's a lot of unknowns. People's careers should not be decided by factors people don't really understand."

Let's unpack this a bit: Bloch's arguing that value-added measures of student learning shouldn't be used for teacher accountability because they're too mathematically complicated, and specifically because teachers shouldn't be held accountable using something they can't understand. Well, let's stipulate that the math beyond the measures is complicated. It would certainly be possible to hold teachers accountable for student learning using very simple and transparent measures--like the percentage of a teacher's students achieving proficiency. But I'm guessing most teachers wouldn't like that, and would rightly argue that such measures are unfair because they fail to take into account the variety of other factors--demographics, prior performance, etc.--that affect student learning. The complexity of value-added models is exactly the result of trying to take these factors into account. As with any complex issue, you have to make trade offs, in this case, between transparency or simplicity and fairness or accuracy. Unless of course, one doesn't think teachers should be held responsible for student learning at all......

That said, I do think reformers who advocate new teacher evaluation systems ought to take more seriously the fact that the new value-added and growth measures are difficult to understand and often not entirely transparent. Too often proponents of new teacher evaluations tend to talk as if value-added or growth measures are a sort of magical black box that, if you put the right data in, will spit out the truth about teachers' effectiveness in the classroom. But it's more complicated than that. As Mr. Weasley said, "Never trust anything that can think for itself if you can't see where it keeps its brain." We shouldn't be asking teachers to place their trust in a magical black box without being transparent about how this all works and the real limitations of the measures.


You must be logged in to leave a comment. Login | Register
Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.

Advertisement

Recent Comments

Archives

Categories

Technorati

Technorati search

» Blogs that link here

Tags

AFT
Alex Grodd
Ana Menezes
Andrew Kelly
appropriations
ARRA
Aspire Public Schools
authorizing
Better Lesson
Bill Ferguson
certification
charter schools
child care
children's literature
choice
civil rights
CLASS
Core Knowledge
curriculum
D.C.
democracy
early childhood
Early Learning Challenge Grant
economics
elections
English language learners
entrepreneurship
equity
Evan Stone
fathers
finance
fix poverty first
Hailly Korman
harlem children's zone
HEA
Head Start
head start
health care
Higher Education
home-based child care
homeschooling
housing
How we think and talk about pre-k evidence
i3
IDEA
income inequality
instruction
international
Jason Chaffetz
Jen Medbery
just for fun
Justin Cohen
Kaya Henderson
Kenya
kindergarten
KIPP
Kirabo Jackson
Kwame Brown
land use
LearnBoost
libertarians
LIFO
literacy
Los Angeles
Louise Stoney
Mark Zuckerberg
Maryland
Massachusetts
Memphis
Michelle Rhee
Michigan
Mickey Muldoon
Neerav Kingsland
New Jersey
New Orleans
NewtownReaction
Next Gen Leaders
Next Gen leaders
nonsense
NSVF Summit
NYT
organizing
parent engagement
parenting
parking
pell grants
politics
poverty
PreK-3rd
presidents
principals
productivity
QRIS
Race to the Top
Rafael Corrales
redshirting
regulation
religion
rick hess
Roxanna Elden
RTT
san francisco
school choice
social services
SOTU
special education
Stephanie Wilson
stimulus
story
Sydney Morris
tax credits
Teacher Prep
teachers
technology
Title I
unions
urban issues
Vincent Gray
vouchers
Waiting for Superman
Washington
West Virginia
zoning