Before NCAA Divisions, We Need Better Data
Yesterday, Alexander Russo applied the concept of NCAA divisions to the comparison group debate. He suggested:
What about creating NCAA-like divisions (I, II, III) within public school systems based on student poverty, in order to help someone (educators) get past the poverty- achievement trap and help others (politicos) see that performance varies even with schools with similar demographics?
The trouble is that public schools only have access to blunt measures of students' socioeconomic status and other non-school conditions. In particular, free and reduced lunch eligibility poorly captures degrees of disadvantage. Imagine two schools in which 60% of students qualify for free lunch. In one school, free lunch qualifiers are from families making 95% of the poverty line; in the second school, these kids are from families earning 50% of the poverty line. With currently available data, we falsely make apples-to-apples comparisons between these schools. By the same token, a school full of poor graduate students' kids can look a lot like one with kids facing multigenerational poverty if we only consider free/reduced lunch measures.
If we want to construct accurate comparison groups, we need to collect additional data on parental education, income, family structure, etc. A massive data collection effort isn't in the stars, though. So when we read sentences like, "School 1's share of students from low-income households is identical to that of School 2, so differences in test scores cannot be attributed to poverty," we should, at the very least, take a closer look. (See these related posts on the no excuses argument or NYC's peer groups).
What about creating NCAA-like divisions (I, II, III) within public school systems based on student poverty, in order to help someone (educators) get past the poverty- achievement trap and help others (politicos) see that performance varies even with schools with similar demographics?
The trouble is that public schools only have access to blunt measures of students' socioeconomic status and other non-school conditions. In particular, free and reduced lunch eligibility poorly captures degrees of disadvantage. Imagine two schools in which 60% of students qualify for free lunch. In one school, free lunch qualifiers are from families making 95% of the poverty line; in the second school, these kids are from families earning 50% of the poverty line. With currently available data, we falsely make apples-to-apples comparisons between these schools. By the same token, a school full of poor graduate students' kids can look a lot like one with kids facing multigenerational poverty if we only consider free/reduced lunch measures.
If we want to construct accurate comparison groups, we need to collect additional data on parental education, income, family structure, etc. A massive data collection effort isn't in the stars, though. So when we read sentences like, "School 1's share of students from low-income households is identical to that of School 2, so differences in test scores cannot be attributed to poverty," we should, at the very least, take a closer look. (See these related posts on the no excuses argument or NYC's peer groups).
