By 2003, the press - and the Texas Education Agency - started looking more closely at Houston's results. In the Times first article on the Houston miracle, "Questions on Data Cloud Luster of Houston Schools," Diana Schemo wrote, "Now, some here are questioning whether the miracle may have been smoke and mirrors, at least on the high school level. And they are suggesting that perhaps Houston is a model of how the focus on school accountability can sometimes go wrong, driving administrators to alter data or push students likely to mar a school's profile -- through poor attendance or low test scores -- out the back door."
Ten days later, the Times editorial page wrote that Paige "owes it to the country to share his thoughts on how this happened and what it means." In an interview with the Times editorial board a few days later, Paige defended his record. Gains in student achievement were real and "still standing," though he said ''there probably was'' a dropout problem.
But the cat was out of the bag. By December the Times had acquired test score data - both on the Texas TAAS and the nationally normed Stanford tests - and established that Houston's state test score gains were enormously inflated. In other words, Houston's sizable gains on the Texas test largely evaporated on the Stanford 9. In August 2004, 60 Minutes ran a segment on the Texas Miracle. When the Dallas Morning News uncovered widespread cheating in Houston late in 2004, it appeared that the game was finally over.
In this month's issue of Educational Evaluation and Policy Analysis, a new study by UT-Austin professor Julian Vasquez-Heilig and Linda Darling-Hammond, "Accountability Texas-Style: The Progress and Learning of Urban Minority Students in a High-Stakes Testing Context," revisits the Houston miracle by analyzing years of student-level test score and graduation data (1995-2002). There's no version up on the web yet, but here are some key findings:
* Growth on scores on TAAS exam outpaced scores on the Stanford exam. This appears to be prima facie evidence of test score inflation.
* Low-scoring students were excluded from taking the TAAS, both through special education and language exemptions and grade retention.
* A key strategy for improving test scores involved retaining students in 9th grade so they would not sit for the TAAS exit exam in 10th grade. At its peak, 30% of 9th graders were retained for one or more years. Some students were kept in 9th grade for two years, and then skipped to 11th grade so they could avoid the exit test. When more students were retained, unsurprisingly, accountability ratings went up.
* While minuscule dropout rates were reported, only a third of students were graduating in Houston in 5 years or less.
Take home lessons? If it looks too good to be true, it probably is.