Timely Tidbits on Unintended Consequences
Freakonomics and Marginal Revolution face off on unintended consequences - it's timely food for thought about the potential consequences of adopting value-added as the primary measure of teacher effectiveness. As I've noted before, value-added as one of many measures works for me; value-added as the master measure - which I fear it would become - does not. Why? Teaching is a multifaceted task, and value-added measures use a simplistic evaluation rubric to monitor a complex task. Alex Tabarrok sums up the potential problem here:
The law of unintended consequences is what happens when a simple system tries to regulate a complex system. The political system is simple, it operates with limited information (rational ignorance), short time horizons, low feedback, and poor and misaligned incentives. Society in contrast is a complex, evolving, high-feedback, incentive-driven system. When a simple system tries to regulate a complex system you often get unintended consequences.
An unintended consequence of blogging is that I am about to miss a deadline, so I've got to bounce. Stay tuned for more value-added debate next week. Enjoy the weekend, everyone!
The law of unintended consequences is what happens when a simple system tries to regulate a complex system. The political system is simple, it operates with limited information (rational ignorance), short time horizons, low feedback, and poor and misaligned incentives. Society in contrast is a complex, evolving, high-feedback, incentive-driven system. When a simple system tries to regulate a complex system you often get unintended consequences.
An unintended consequence of blogging is that I am about to miss a deadline, so I've got to bounce. Stay tuned for more value-added debate next week. Enjoy the weekend, everyone!


Comments
Take inventory of the practical and social science wisdom of the Eduwonkette since January 8, and what do you get?
The first post was a concise summary of the “insignificant effects of schools vs. teachers” in counteracting the effects of poverty.
Perhaps explaining our disappointing results of school reform is, “The law of unintended consequences is what happens when a simple system tries to regulate a complex system.”
Ironically, one of most important points was made but the only commentator who was completely inappropriate, but who did make the accurate statement that, “the actual work of teaching is more complex than heart surgery.
At any rate, “‘the bottom line is clear, test-based accountability has not generated significant gains in student achievement.’”
The Wonkette linked to a summary of “the issue” that explains the failure, “kids who can decode but not comprehend,” and the “deleterious impact of the testing culture.”
She also linked to the crucial ingredient to successful reform is “trust.” Firstly, reforms can not succeed without buy-in, and NYC by “violating the tenets of research ethics undermines trust.” NYC ignores the need for findings to be “both practically and statistically significant.” Worse, NYC will brag about the results of breaking up a 900 student school into five small schools that serve 540 students, but ignore the 460 students it did not cream off, and it never asks, “where did the displaced kids go?” (A commentator supplemented the Wonkette’s good sense by noting that deaths due to gunshots in some cities are 33% of the death rate in Iraq, and up to 50% of students in some schools do not live with either parent.)
Also included is plenty of teachers’ wisdom, such as “They never say ‘Thanks for Improving My Test Scores.” She also links to the wisdom of principals, such as “when you add a new task to your teachers’ plates, then something else HAS GOT TO GO.”
Finally, she cites an excellent shorthand for discussing future reforms, contrasting “instructionists vs “incentivists.”
How many educators would have problems with any of those observations? Taken together, don’t they point unmistakably to a much better path?
Posted by: john thompson | January 25, 2008 5:17 PM
I wasn't sure what "value added" actually meant in an educational context, so I dug around for some information. Its use in education dates back at least as far as John Garrett's 1972 book The Management of Government. According to Aidan Rose (2003), "Garrett provides an early discussion about the concept of ‘value added’ in education. At this time, 1970, it was concluded that the focus was on intermediate objectives (in this case pupil-teacher ratios) as opposed to final objectives (educational standards)." This seems remote from the DoE's definition.
It appears that "value added" has a myriad of possible meanings (Fisher and Twing, 2004) in education and in other fields. Which meaning has the DoE chosen, and why? Why, moreover, would the DoE espouse such a murky and clunky term, when something clearer and more euphonic would do?
Posted by: Diana | January 27, 2008 8:36 PM