Denver's ProComp pay program may have helped attract more-effective teachers to the district and boosted retention in hard-to-serve schools, according to a report on the much-discussed system released recently by the University of Colorado at Boulder. Teachers opting into the program also appear to be slightly more effective on the whole.
The analysis was based on student and teacher data from eight school years, from 2001-02 through 2008-09. (ProComp began in 2005-06 , with opt-in periods for teachers in 2006, 2007, 2008, and 2009.)
Researchers compared each student's results with those of other students with similar achievement histories and traced the data back to the students' teachers. They also examined value-added data linked to specific teachers. And they reviewed data from surveys of teachers and principals in the district.
Here's a rundown of the findings:
— Student achievement improved over the time period studied, but all teachers appear to have seen increased achievement in the years subsequent to implementation and there was little evidence to indicate that those who opted into ProComp were more "productive" on average at boosting achievement than those who did not.
— Teachers who opted into ProComp do appear to have slightly outperformed their peers, but it's not clear whether this is because of ProComp itself or because of the self-selection of more-effective teachers to join the program.
— Teachers hired after ProComp went into effect consistently show higher first-year achievement in both reading and math than those hired prior to the program. The findings persist for those teachers' first three years, suggesting that ProComp may have served to attracted more able teachers to the district.
— Retention rates generally increased over the time period studied; schools with median levels of teachers in ProComp had higher rates of retention overall.
— Hard-to-serve schools with higher levels of teachers participating in ProComp experienced a sharp increase in retention rates in the first full year of ProComp and the greatest increase in retention, from 74 percent to 86 percent, again suggesting a possible ProComp effect.
Now, I know you're all wondering how this jives with that other big performance-pay study that came out recently, on the Chicago Teacher Advancement Program, which had no positive findings.
But it's important to note that these aren't comparable studies, and that neither is conclusive about the impact of performance pay. One of the reasons is because of major differences in how the programs were studied. The Chicago study used an experimental research design, allowing us to draw some cause-and-effect conclusions.
This Denver study is based mostly on observational data. That means that ProComp, student scores, and teacher hiring may be related, but the data don't say definitively whether the program actually caused those changes.
It is also worth pointing out that, like TAP, ProComp is far more than a performance-pay program. Both programs also include professional-development aspects and pay for other aspectsin TAP for taking on additional roles and responsibilities, and in ProComp for taking advanced coursework, teaching in hard-to-staff subjects, and working in high-need schools and fields.
The individual teacher-pay component of ProComp based on student-achievement results is less heavily emphasized than in TAP, boosting salaries only a little more over 7 percent of the base-pay level. (Compare that with 9 percent for advanced degrees and about 13 percent for incentives to take on the tougher assignments.)
In effect, with both studies, it's hard to determine exactly how bonuses for advancing student achievement might have affected the field separate and apart from all the other moving pieces.
The authors of this report note that an upcoming study will describe outcomes "at a finer level of granularity to better understand differential outcomes of the program's various elements," so let's hope more is coming our way on this. Stay tuned.