It’s hard to make sense of all the differing findings coming out of new studies on charter schools. A couple of studies suggest that students in charter schools fare no better than their peers in more-traditional public school settings, while others find that charters have a positive effect on student learning. Which conclusion is right?
Mathematica researcher Brian Gill attempted to address that difficult question in a webinar hosted Friday by the research group’s Center for Improving Research Evidence. Gill sees some reason for hope in the national randomized study on charter schools that was released last month by the federal Institute of Education Sciences. (Mathematica also conducted that study.)
“Before this new IES national study came out, it would have been possible to take the view that the differences were attributable to methodological differences in the studies,” Gill said in a telephone conversation after the webinar. The studies that assigned students to experimental and control groups based on admissions lotteries tended to find positive effects, while the nonexperimental studies yielded more mixed results. The Mathematica study broke the pattern, however, because it was a randomized study that found charter pupils in general were no more successful in school than their counterparts in regular public schools.
Rather than muddle the research picture, Gill said the Mathematica findings pointed up some areas in which the research seems to be converging. For instance, several studies show that charters are more successful with disadvantaged students in urban school systems and less effective with suburban populations. (Mike Petrilli makes a similar observation this month in this post.)
“From the perspective of a researcher, that’s encouraging,” Gill said, “and it suggests that maybe, in fact, we can conduct nonexperimental studies that can replicate the experimental findings.” That’s important, Gill says, because it’s really hard to find enough oversubscribed charters to do large, credible randomized studies. And you have to wonder, if you’re only studying schools that people are banging on the door to get into, whether you’re getting an accurate picture of what’s going on in the full range of schools that exist.
“As valuable as the admissions-lottery studies are, we need more than just that,” he said.
The consistent findings on charter schools’ effectiveness with disadvantaged, urban populations—and their apparent ineffectiveness in suburbia—also made Gill wonder why middle-class parents continue to choose them. “One thought I had was that maybe middle-class families are looking for something other than test scores,” he said. (Petrilli raises this question, too, in his analysis.)
“I think the findings tend to reinforce the point that a lot people are making, which is that we need to have more and richer measures of school performance,” Gill added. The recent final results from the District of Columbia voucher program suggest a case in point: Students in those schools didn’t experience any dramatic gains in test scores, but college-going rates saw a big jump.
Gill says the next generation of studies on charter schools is now taking a closer look under the hood to see what effective charter schools might be doing differently from everyone else. “In a sense, doing that kind of research can help us move in the direction of fulfilling one of key purposes of charter schools, which is to serve as laboratories of innovation,” he said.