Review Turns up Faulty Data on Two More 'Best High Schools'
The National Center for Education Statistics, which maintains a repository of information on all of the country's public schools and districts through its Common Core of Data, has uncovered faulty data on two additional schools in Arizona and Pennsylvania that made it into U.S. News and World Report's rankings of the nation's best high schools.
The magazine, which relies in part on data collected by the federal government, listed Estrella Foothills High School in Goodyear, Ariz., as the sixth best in the state out of 118 ranked, with an enrollment of 198 students and 51 teachers. On its website, the high school is displaying a gold medallion marking the honor. But in actuality, the school has an enrollment of about 1,000 students, and the student-teacher ratio is 26 to 1, said a district representative. The superintendent, Beverly J. Hurley, was on a retreat and could not be reached for comment Wednesday. Enrollment information was also incorrect for the other two high schools in this district, though they were unranked by U.S. News and World Report.
Great Valley High School in Malvern, Pa., is ranked 12 out of 193 schools ranked in Pennsylvania. The school was listed as having 30 teachers, which federal statisticians now say is incorrect. Based on that number, the school would have a student-teacher ratio of about 41 students per teacher, much higher than the state's average. But the school actually has an average of about 24 students per class, according to its website. Alan J. Lonoconus, the superintendent of the district, was out of the office Wednesday and could not be reached for comment.
The federal education statistics agency also noted problems with data collected on elementary and middle schools in the Great Valley district, but because those schools aren't high schools, they were not a part of the U.S. News report.
NCES is maintaining an errata report where it is keeping a running tally of schools with questionable data at the top of this webpage. The page also has a letter from Jack Buckley, the commissioner of education statistics, explaining what the agency is doing to root out any additional anomalies.
Inaccuracies Found Early
Questions about the data were raised within a day after the magazine released its May 8 report, which measures high schools on factors such as Advanced Placement pass rates and academic performance among minority and economically disadvantaged students. (You can find the magazine's full methodology here.)
Previously reported errors include one Nevada high school, ranked 13th in the nation, for which federal statistics had mistakenly said the school had 477 students, 111 teachers, and a 100 percent passing rate on AP tests. In actuality, the school has about 2,850 students, a student-teacher ratio that is closer to 24 to 1, and an AP pass rate of about 64 percent. One California high school was listed in the magazine as having 493 students, a 100 percent AP pass rate, and a student-teacher ratio of 7 to 1, based in part on federal data. The truth: The school has 1,650 students and 24 students for every teacher.
Marilyn M. Seastrom, the chief statistician for the NCES, traced those errors back to faulty data that was not caught by computer software designed to check for major anomalies. She said the agency would investigate the data for all 4,877 high schools that were ranked by the magazine. Some more schools are expected to be posted to the errata report by the end of this week, she said.
Brian Kelly, the editor of U.S. News and World Report, said in an interview with Education Week that the magazine was committed to accurate data, and that updated charts reflecting accurate numbers would be available within "a few days."
Those who work with the Common Core of Data frequently recognize that the data can have some glitches, said Brian M. Stecher, an associate director at RAND Corp's education division, a research giant in the field. RAND does use the federal data, but usually to draw a sample of schools and districts for further study. "We often refer to the CCD as a starting point," he said.
But Stecher said a larger issue is the worth of ranking high schools at all. Colleges and universities—which are also rated by U.S. News in well-read reports each year—potentially draw students from all over the country. But people don't generally move around the country to attend top-rated high schools, he said. In addition, high schools differ so much from state to state in important ways, such as in access to AP courses or credits needed to earn a high school diploma. The rankings also can't get at non-academic factors that make a high school attractive to students.
"It just seems to me this is part of this U.S. obsession with ranking everything, and then ignoring everything that isn't number one," Stecher said.
More Rankings to Come
But Kelly, the editor of the magazine, disagrees. The problems turned up in this year's rankings have not prompted the magazine to change its methodology or to pull back on ranking schools, he said. In fact, an additional list, ranking the nation's best science, technology, engineering and math schools will be released soon, he said. In that case, the magazine will be using data such as pass rates on AP tests, and will not be relying on the Common Core of Data, he said.
People do use the data to make real estate decisions, and policymakers within a county or state can evaluate the rankings to find out why similar-situated schools are achieving dramatically different results, he said. Even comparisons among schools in different states has value, he contends.
"You essentially have states that are fooling themselves," Kelly said. "We see some states that are much more represented than others on the best high school list. Why is that?"
He added that rankings are growing in importance, not lessening. "You don't know where you're going unless you have a map, and data helps provide that map." And indeed, U.S. News is not alone: On Monday, Newsweek released its list of the nation's 1,000 best high schools. Rather than relying on federal data, Newsweek's ratings were developed from information gathered from the 2,300 school administrators who responded to the magazine's survey.