The results of educational studies conducted by university researchers are widely accepted as gospel by the vast majority of the public. After all, who should know more about education than educators? But when findings cannot be reproduced, credibility is undermined (“The Debate: Does It Help To Repeat Past Studies? The Wall Street Journal, Oct. 6). This has led to the possibility of a new era in scientific publishing, where self-examination across all fields of inquiry is routine (“The limitations and perils of scientific studies,” Los Angeles Times, Oct. 11).
I’m not suggesting that fraud is widespread, nor am I urging that social science studies be eliminated because results cannot always be replicated (“Psychology Is Not in Crisis,” The New York Times, Sept. 1). But because public education is now a hot issue, professors see an opportunity to make a name for themselves by publishing attention-grabbing findings. Sometimes this means massaging data, which can take many forms. That’s why whenever I read about results that seem too good to be true, I immediately go into skeptic mode. I wrote about this before (“Limitations of Education Studies,” Education Week, Jul. 20, 2012).
For example, studies about school districts that report dramatic increases in test scores virtually overnight are automatically suspect. I tend to believe that they have excluded outliers in order to make themselves look good. Or perhaps data have first been examined and then a report has been written that leaves the impression the experiment discovered precisely the same results.
I remember vividly the so-called Texas miracle that subsequently turned out to be a mirage. Today, charter schools are the subject of claims that I believe will be found to be not quite as dramatic as their advocates claim. I’m not saying that some charter schools don’t deserve high praise for their accomplishments. But what usually is the case is that only the most successful ones make the news, leaving taxpayers with the erroneous belief that all are superior to traditional public schools.
I don’t see an end to questionable investigative practices because pressure to improve public schools is relentless. There will always be those with hidden agendas that seek to use research solely as a way of confirming their pre-existing ideology. That’s why I urge readers to be on the alert. But more important, I hope that educational journals will be more selective in what they choose to publish. Guidelines are needed that make it easier for data to be analyzed.