A Blogger Writes on 'Lies, Damned Lies, and Statistics'
I, like many wonks, am a big fan of educational of educational data. But, in today's blog entry at GothamSchools, blogger Aaron Pallas does a masterful job of showing just how misleading educational data can sometimes be.
Pallas' beef is with the burgeoning number of data-based Web sites that allow users to compare schools. The problem with such comparisons, he says, is that they don't tell parents much about how their children will fare in a particular school.
For one reason: Achievement-wise, there is far more variability from classroom to classroom within the same school than there is from school to school. A school's overall proficiency score will mask those differences.
Pallas, a professor of sociology and education at Teachers College, Columbia University, also notes that the Web ratings typically don't account for differences in the demographics of the student populations that schools serve. He writes:
Since we know that there is a powerful association between family economic status and student achievement, schools serving high concentrations of poor children will, on average, rank lower than schools that service a predominantly middle- or upper-class population. Stating this is not, I believe, a case of the soft bigotry of low expectations. Rather, it's an acknowledgement that a school's context matters in judging how well the school is serving its students.
Researchers, of course, already know this stuff. Home buyers and real estate agents, on the other hand, might want to take note.