Despite the complete lack of evidence that they work, pay-for-performance schemes seem to be as popular as ever. Why does anyone think such plans will improve student learning?
I came across a decent explanation recently. In their 2007 article “What to Do? The Effects of Discrepancies, Incentives, and Time on Dynamic Goal Prioritization” in the Journal of Applied Psychology, Aaron Schmidt and Richard DeShon use a computer-based experiment to explore how performance goals and incentives influence the allocation of resources.
They found, not surprisingly, that when given two competing priorities, people tend to balance resources (in the experiment, time) between them in such a way that they accomplish both goals. Moreover, people tend to allocate more resources to the priority with the greatest discrepancy between the goal and the current level of performance. In other words, people tend to focus more on areas where they need to catch up. When it starts to seem impossible to achieve both goals, e.g. when they are running out of time, people shift their energy to the goal they are most likely to accomplish.
The study also explored the influence of incentives. As you might expect, when one of the tasks has an incentive tied to its completion, that task tends to get done more often than the other, and when this incentive is framed negatively—as a sanction—rather than as a reward, people tend to reach the goal more often. If we tie incentives to both goals, people will achieve both goals at higher rates than when there are no incentives.
So far, this seems like a solid case for NCLB-style accountability measures. If we set targets and sanctions, it seems plausible that schools will improve in the areas in which they are behind.
The problem, though, is that teaching is nothing like the task demanded of participants in Schmidt and DeShon’s study. Yes, educators have multiple competing priorities, but the key word is multiple—not just a few that can be easily juggled in order to ensure that all goals are met. Teaching is highly complex, far more so than the simple computerized scheduling task that was the basis for the study, and teachers are already motivated.
Schmidt and DeShon seem confident enough in their operationalization of prioritization, incentives ($10 movie tickets), and performance to treat their findings as general principles of human behavior. Nevertheless, they warn of using incentives cavalierly:
Another important extension regarding this single-incentive effect is the explicit consideration of the effects that incentives for one performance domain may have on other dimensions of performance. This multiple-goal approach revealed that increased focus on a rewarded performance domain came at the expense of the unrewarded domain. Further, siphoning of resources from unrewarded tasks occurred even when equivalent performance goals were provided for each task. This suggests that great care is required when implementing incentive structures to avoid unintended decrements in other performance domains.
In other words, Campbell’s Law is a very real concern. We can’t measure everything, and we therefore can’t incentivize every goal we have for public education. When we tie incentives to only a few goals, we narrow the curriculum—the opposite of enrichment. Even if we could develop a perfectly balanced system of goals and incentives, we’d find that teaching is far too complex to be improved by so simplistic a mechanism.