The What Works Clearinghouse, an initiative of the federal Institute of Education Sciences, said today that a widely publicized study on Philadelphia’s “Renaissance Schools” model did not meet its standards for evidence because the Renaissance schools were not measured against a set of schools with similar achievement levels.
That ranking is the lowest that the clearinghouse can give a research report; the other ratings are “meets evidence standards” or “meets evidence standards with reservations.”
In response to the rating, Research for Action, the Philadelphia-based organization that authored the report , said that it is sticking by its conclusion that early results showed positive academic results among the 11 K-8 Renaissance schools. All were low-performing schools that were either turned over to charter managers or placed under direct district management. The district-run schools were called Promise Academies. The changes at the schools usually included a longer day and year and reconstitution of the school staff. The charter-run schools were allowed to operate with some autonomy from the district.
The study found that the two Renaissance high schools studied did not show discernible academic or attendance gains.
The study is “a pretty rigorous design,” said Kate Shaw, the executive director of Research for Action. “What it shows is that something is happening in Philadelphia’s Renaissance schools.”
The clearinghouse is attempting to get out more quick reviews of studies that are in the news. In this case, the study received a great deal of attention in Philadelphia, where the cash-strapped district of 146,000 students is trying to decide how and if it can expand its reform agenda. I also wrote a blog post about the study in February.
Shaw said that the researchers with her organization could not pick a group of schools that were exactly like the Renaissance schools, because all the district’s lowest-performing schools were selected for reform measures. Instead, the researchers selected a group of low-performing schools that closely matched the Renaissance schools in demographics and academic performance. The researchers analyzed data going back five years to see if the performance of the Renaissance schools and the comparison group were similar.
“These schools were all improving at the same very low rate,” Shaw said. But then after the Renaissance initiative, “you saw a very distinct change in the gains” among the schools chosen for the reform measures.
Shaw noted that the What Works Clearinghouse is known for having strict standards for research, which Education Week explored in an article about the clearinghouse trying to shed its “nothing works” image. Meeting those strict standards is difficult in a school system, which is not trying to conduct research but to improve its schools, she said.
The organization is working to secure funding to continue its research, Shaw said.