« Guest Blogger Mike Klonsky: The Small Schools Movement Meets the Ownership Society | Main | Guest Blogger Mike Klonsky Part II: Deb Meier’s Innovation Became Bloomberg’s Bulldozer »

What Can Other Professions Teach Us about Evaluation and Accountability in Education?

In a very productive exchange, Dean Millot and Corey Bower have been contemplating the professional status of education. Dean's most recent post, "Why Legally Recognized Professionalism is Necessary to Reasonable Teacher Accountability," is one of the best think pieces I've read in some time. Read the whole thing, but here's the central theme of the post:

Lawyers and doctors are not punished for undesired outcomes; they are accountable for doing what professionals should do given their client’s circumstances....As a legally recognized profession, teacher conduct would be judged by teachers, according to standards of educational care devised by teachers, applied to the client circumstances in question.

Dean's post links well with AEFA conference talks by Randi Weingarten and Richard Rothstein last weekend. Weingarten also drew on the medical metaphor to argue that "teachers are physicians of the mind." In her view, there is a difference between the most skilled physician and a miracle worker. Just as the best hospitals can't solve public health crises on their own, Weingarten argued that, "schools cannot beat back all personal, social, and economic challenges that kids have." In an op-ed last week, she also endorsed a professional standard similar to that proposed by Dean:

[Teachers] should be assessed on how they use test scores and other data to adjust their teaching to help students improve....The approach is akin to judging doctors on how they use the results of blood tests, X-rays, and the like to prescribe a course of treatment.

In his talk, Rothstein drew on the experience of more fields than I can name (business, medicine, public works, etc). Despite many leaders' calls for education to mimic the private sector, Rothstein's review concluded that "private sector performance incentives rely primarily on subjective evaluations, not easily corrupted quantitative measurements." The central theme of the talk was that systems of measurement distort the processes they are intended to measure. The paper on which the talk is based - "Holding Accountability to Account: How Scholarship and Experience in Other Fields Inform Exploration of Performance Incentives in Education" - is a comparative/historical tour de force, and a must read if you're interested in the evaluation question.

Blog posts without positions generally fall on their face, but I still have more questions than answers about Dean's proposal. Here are the two questions I'm pondering:

* How do the processes of diagnosis, inference, and treatment in education differ from those in medicine and law, and what are the implications of these differences for "professional accountability?"

* How does the state of our knowledge about educational diagnosis and treatment differ from that in other professions?

Constructed properly, I don’t think this analogy serves Randi well at all. If doctors are like teachers, then hospitals are like schools. Hospitals are officially evaluated in a couple of ways. First, they must be accredited, which is basically an analysis of inputs (staffing and equipment levels, facilities maintenance etc) but not a judgment about the quality of care. This is more or less the way schools have been run until recently.

But hospitals are also evaluated for outcomes and quality of care by, among others, the federal and state governments (via Medicare/Medicaid) and by insurance companies. They look at things like length of patient stays, illness caused improper or inadequate care within the hospital, and mortality rates. They break out the analyses for different conditions and procedures, and attempt to adjust for the age and health of patients when they enter. They compare hospitals against one another and against norms. This is something like the way NCLB–driven accountability systems have been working so far.

Hospitals that perform poorly on their quality-of-care reviews face a number of sanctions. They can be dropped from insurance company payment networks. They can de denied government reimbursement for the care they render. Monitors can be imposed. Perhaps most importantly, the public is encouraged to consider these ratings before choosing a provider.

You can be pretty sure that a hospital facing sanctions for, say, bad cardiac outcomes will look pretty closely at the success rates of its individual cardiologists. If they find that a few are contributing disproportionately to high rates of mortality, those doctors will lose the right to operate at that hospital. I suspect that most parents would be comfortable with an analogous regime at their school. Would Randi?

Certainly, measurement and goal-setting leads to predictable behavioral distortions, as per Rothstein. But so does the lack of measurement and goal-setting.

I have to disagree on the doctor/teacher analogy. Doctors are paid a heck of a lot more than teachers. They police their own field. They're treated like professionals by others in their workplace. They're given the autonomy with which to make the correct decisions. I don't think unions would object much to thorough evaluations of teachers and schools under those conditions.

I think it’s mis-focused to think of this issue in terms of teachers and doctors as independent contractors. It’s schools that are the accountable institutions, and so what’s framed as “teacher accountability” is really about the rights of the managers of those institutions (principals, superintendents) to evaluate, compensate, and, sometimes, terminate their staff based on outcomes, or any other criteria they choose.

The debate about whether teachers should by measured and judged by test scores is in some ways a distraction from the more fundamental (and, I think, a more threatening) question of why school managers shouldn’t have the right to choose and manage their employees with the same freedom that every other enterprise has.

Employees at law firms and restaurants, hospitals and steel factories, architects and hardware stores, investment banks and landscapers—pretty much every professional, managerial, and blue-collar worker in the country—take this for granted. Those who don’t go into private practice or become independent contractors, where they are even more at the mercy of the whims of their customers. Whether or not many public school teachers would choose that path is a moot question, since it’s not an option.

There are some critical differences between doctors and teachers that never seem to come up (at least when teachers are driving the analogy). No doctor would ever be turned loose to make independent decisions on patients at the level of a bachelor's degree. In fact the bachelor's degree is generally just the beginning of the road moving through acceptance in to medical school, extended (and expensive) education, internship, residency, etc. Doctors are also held personally and individually responsible for the medical decisions that they make. How many lawsuits are there against teachers who don't succeed in teaching kids how to read? How much are teachers paying for malpractice insurance?

In the end, I don't think teachers really want to be treated like doctors.

Medical science is mostly sound. Education science is a joke. Doctors must treat using the Standard of Care, based on scientific research that has been proven to be statistically sound. Teachers use best practices that change depending on student, and have no science-based predictive outcomes other than a particular lesson working or not for a particular student or another; the very definition of anecdotal.

The problem with education is not evaluation of teachers, or comparisons that may or may not make sense. The problem is that we don't value education much as a society, and we expect teachers to solve problems that they cannot solve. Like a doctor who cannot cure your severed arm. Try and not sever it in the first place; and don't blame the doctor if she can't re-attach it!

Comments are now closed for this post.


Recent Comments

  • rishy: Medical science is mostly sound. Education science is a joke. read more
  • Margo/Mom: There are some critical differences between doctors and teachers that read more
  • Steven Hodas: I think it’s mis-focused to think of this issue in read more
  • Corey: I have to disagree on the doctor/teacher analogy. Doctors are read more
  • Steven Hodas: Constructed properly, I don’t think this analogy serves Randi well read more




Technorati search

» Blogs that link here


8th grade retention
Fordham Foundation
The New Teacher Project
Tim Daly
absent teacher reserve
absent teacher reserve

accountability in Texas
accountability systems in education
achievement gap
achievement gap in New York City
acting white
AERA annual meetings
AERA conference
Alexander Russo
Algebra II
American Association of University Women
American Education Research Associatio
American Education Research Association
American Educational Research Journal
American Federation of Teachers
Andrew Ho
Art Siebens
Baltimore City Public Schools
Barack Obama
Bill Ayers
black-white achievement gap
books on educational research
boy crisis
brain-based education
Brian Jacob
bubble kids
Building on the Basics
Cambridge Education
carnival of education
Caroline Hoxby
Caroline Hoxby charter schools
cell phone plan
charter schools
Checker Finn
Chicago shooting
Chicago violence
Chris Cerf
class size
Coby Loup
college access
cool people you should know
credit recovery
curriculum narrowing
Dan Willingham
data driven
data-driven decision making
data-driven decision-making
David Cantor
Dean Millot
demographics of schoolchildren
Department of Assessment and Accountability
Department of Education budget
Diplomas Count
disadvantages of elite education
do schools matter
Doug Ready
Doug Staiger
dropout factories
dropout rate
education books
education policy
education policy thinktanks
educational equity
educational research
educational triage
effects of neighborhoods on education
effects of No Child Left Behind
effects of schools
effects of Teach for America
elite education
Everyday Antiracism
excessed teachers
exit exams
experienced teachers
Fordham and Ogbu
Fordham Foundation
Frederick Douglass High School
Gates Foundation
gender and education
gender and math
gender and science and mathematics
gifted and talented
gifted and talented admissions
gifted and talented program
gifted and talented programs in New York City
girls and math
good schools
graduate student union
graduation rate
graduation rates
guns in Chicago
health benefits for teachers
High Achievers
high school
high school dropouts
high school exit exams
high school graduates
high school graduation rate
high-stakes testing
high-stakes tests and science
higher ed
higher education
highly effective teachers
Houston Independent School District
how to choose a school
incentives in education
Institute for Education Sciences
is teaching a profession?
is the No Child Left Behind Act working
Jay Greene
Jim Liebman
Joel Klein
John Merrow
Jonah Rockoff
Kevin Carey
KIPP and boys
KIPP and gender
Lake Woebegon
Lars Lefgren
leaving teaching
Leonard Sax
Liam Julian

Marcus Winters
math achievement for girls
meaning of high school diploma
Mica Pollock
Michael Bloomberg
Michelle Rhee
Michelle Rhee teacher contract
Mike Bloomberg
Mike Klonsky
Mike Petrilli
narrowing the curriculum
National Center for Education Statistics Condition of Education
new teachers
New York City
New York City bonuses for principals
New York City budget
New York City budget cuts
New York City Budget cuts
New York City Department of Education
New York City Department of Education Truth Squad
New York City ELA and Math Results 2008
New York City gifted and talented
New York City Progress Report
New York City Quality Review
New York City school budget cuts
New York City school closing
New York City schools
New York City small schools
New York City social promotion
New York City teacher experiment
New York City teacher salaries
New York City teacher tenure
New York City Test scores 2008
New York City value-added
New York State ELA and Math 2008
New York State ELA and Math Results 2008
New York State ELA and Math Scores 2008
New York State ELA Exam
New York state ELA test
New York State Test scores
No Child Left Behind
No Child Left Behind Act
passing rates
picking a school
press office
principal bonuses
proficiency scores
push outs
qualitative educational research
qualitative research in education
quitting teaching
race and education
racial segregation in schools
Randall Reback
Randi Weingarten
Randy Reback
recovering credits in high school
Rick Hess
Robert Balfanz
Robert Pondiscio
Roland Fryer
Russ Whitehurst
Sarah Reckhow
school budget cuts in New York City
school choice
school effects
school integration
single sex education
small schools
small schools in New York City
social justice teaching
Sol Stern
Stefanie DeLuca
stereotype threat
talented and gifted
talking about race
talking about race in schools
Teach for America
teacher effectiveness
teacher effects
teacher quailty
teacher quality
teacher tenure
teachers and obesity
Teachers College
teachers versus doctors
teaching as career
teaching for social justice
teaching profession
test score inflation
test scores
test scores in New York City
testing and accountability
Texas accountability
The No Child Left Behind Act
The Persistence of Teacher-Induced Learning Gains
thinktanks in educational research
Thomas B. Fordham Foundation
Tom Kane
University of Iowa
Urban Institute study of Teach for America
Urban Institute Teach for America
value-added assessment
Wendy Kopp
women and graduate school science and engineering
women and science
women in math and science
Woodrow Wilson High School