Are Racial Achievement Gaps Closing in Chicago?
I promise that this whole week won't be deadly depressing, but Alexander Russo threw down the gauntlet about the media's lack of attention to results in Chicago Public Schools under Arne Duncan. So I took a look at Chicago's NAEP performance.
Have gaps separating white/black and white/Hispanic students in Chicago shrunk in the last 5-6 years?
Nah.
There are no statistically significant declines in these gaps in 4th or 8th grade reading or math. In many cases - for example, 4th and 8th grade math and 8th grade reading - it's not that the black-white achievement gap is declining, but not by enough to be statistically significant. These gaps are actually growing. Sigh.
Lucky for Duncan, Joel Klein doesn't have a leg up in this area - neither superintendent has been successful in narrowing racial achievement gaps. (See NYC achievement gap analyses in painful detail here.)
And while we're on the topic of Joel Klein and NAEP scores, a word on his latest statistical gaffe about sampling. From the outback, here's the exchange between a reporter and Klein at his Australian National Press Club talk:
Have gaps separating white/black and white/Hispanic students in Chicago shrunk in the last 5-6 years?
Nah.
There are no statistically significant declines in these gaps in 4th or 8th grade reading or math. In many cases - for example, 4th and 8th grade math and 8th grade reading - it's not that the black-white achievement gap is declining, but not by enough to be statistically significant. These gaps are actually growing. Sigh.
Lucky for Duncan, Joel Klein doesn't have a leg up in this area - neither superintendent has been successful in narrowing racial achievement gaps. (See NYC achievement gap analyses in painful detail here.)
And while we're on the topic of Joel Klein and NAEP scores, a word on his latest statistical gaffe about sampling. From the outback, here's the exchange between a reporter and Klein at his Australian National Press Club talk:
Reporter: Are you concerned that the only independent national assessment of American children shows that there is actually no improvement in New York, with the slight exception of year four maths?Given that a sample is appropriately drawn - and there is no reason to believe that the NYC NAEP sample is not - it should produce unbiased estimates of the achievement of the underlying population, NYC kids. This guy is being discussed as a candidate for Secretary of Education, and in the same breath telling Australians that our national test is inaccurate because it's based on a sample. Oy.
Klein: [The NAEP tests] are sampled, whereas our state tests are mandatory. They're the accountability and, if you look at the data that I provided to you, across the board those state tests show that we're outperforming the rest of the state and everyone else.


Comments
If scores are rising for all groups, then the fact that the gap is slightly larger now than in the past is not such a concern.
Using the 2007 grade 4 data, I see that 47% of non-Hispanic whites are at or above "proficient" in math and 40% in reading, compared to 31% and 37% in 2003. For blacks, the percents are 8% and 10% vs. 4% and 10%. For Hispanics, the percents are 16% and 14% vs. 10% and 12%. So the rising tide *IS* lifting all boats, just some faster than others.
I'd like to see a greater focus on raising achievement levels for ALL children rather than only focusing on certain groups.
Posted by: Crimson Wife | December 9, 2008 11:31 AM
Gotta disagree with your claim that if all groups' scores are rising, the gap is not a big concern, Crimson Wife. eduwonkette's and my reasoning on this is laid out here.
Posted by: skoolboy | December 9, 2008 12:13 PM
thanks for taking up the challenge, jennifer --
worth noting also that duncan was one of the most publicly resistant and disparaging about NCLB accountability and hqt and tutoring requirements back in the day.
-- alexander
Posted by: alexander | December 9, 2008 1:31 PM
Just wanted to throw in that TIMSS scores were just released--it looks as though there may be some areas where gaps might be narrowing slightly--would love to hear your take on it.
Posted by: Margo/Mom | December 9, 2008 4:51 PM
Don't hold your breath, Margo.
Did you ever notice how Eduwonkette disparages tests, except when citing tests scores suits her political purposes?
Posted by: Charles Barone | December 9, 2008 7:03 PM
Hi everyone,
Crimson Wife - I'll echo skoolboy on this point - if we want key social institutions to reflect the demographic makeup of the population, gaps need to close (and of course we like it when scores increase, too).
Margo/Mom - Definitely will take a look at the TIMSS data later this week - am always happy to hear about narrowing gaps.
Charlie - Thanks for stopping by. I don't see myself as someone who disparages tests. My position is that it's important that we recognize what tests can and can't tell us, and be careful about the inferences we make from them - but I certainly don't believe that they have no place in public education or anything extreme like that.
Posted by: eduwonkette | December 9, 2008 8:53 PM
On Superintendent Klein giving his version of the law of large numbers and the disparagement of samples:
When the selection of favorable statistics to report for argument is as commonplace as the reporting of a selection of favorable anecdotes as "statistics" -- of course the former litigator is going to appeal to what he means by the "totality of the data" (and, in his mind, "the evidence".)
An observation about measurement in different contexts is that sometimes sampling is the only way to achieve high quality." I believe it because I too often must work with error-ridden administrative records; and because the great sampling statisticians all include in their textbooks that argument. IOW, you sample not because you don't have resources for a census, but sometimes because the estimate from the well-designed and executed sample will be better than what is possible from a census. Sampling error will be small relative to data, procedure, and model error. This is all counter-intuitive and must be taught and demonstrated. Most folks believe that accounting audits are complete and to the penny, when for the reasons given --quality control --they are most often based on samples.
Substance: When Crimson Wife cites compares differences in performance over a 4 year time span, I ask: 1) Are these real? 2) Are they for the comparable ethnically-identified populations? And most importantly: 3) At this rate, how long will it take for certain groups to achieve competence? 50 years of uninterrupted improvement?
Incredulous in DC
Posted by: Incredulous | December 10, 2008 2:23 PM
Sorry, but I don't see the goal of education to be ending the socioeconomic inequalities in our society. There are too many cultural factors contributing to those, such as the tremendous variation in the percentage of out-of-wedlock births between different groups (70% for African-Americans, 48% for Latinas, but only 25% for non-Latina whites).
The goal of education should be to provide equality of ACCESS (something that is unfortunately still lacking in our society) rather than equality of OUCTOME (something that is impossible anyways).
Children should receive schooling that helps them reach their own individual maximum potentials. It appears to me that Chicago is making some amount of progress in that direction.
Posted by: Crimson Wife | December 10, 2008 7:06 PM
"ouctome" should read "outcome", sorry for the typo!
Posted by: Crimson Wife | December 10, 2008 7:07 PM
cw:
There is something appalling in using the ACCESS vs OUTCOME shell game to justify a society in which the fruits are clearly maldistributed. It harkens back to the fallacies of separate but equal. It turns out that not only was separate never equal (in terms of any countable input--not that anyone WAS counting), but the notion of separation itself communicated a notion of inequality that could not be undone by simple equality of resources--had that been the case.
I don't know of any particularly meaningful measures of ACCESS--but I can guarantee they are not a part of any accountability system currently in use. But the application of the "you can lead a horse to water..." school of thinking leads us to stop asking questions about why the horse is not drinking. Horses, as other biological entities, have an innate desire to take on water as needed. When a horse does not do so--in the presence of water--something is wrong, with the horse or the water. It behooves us to care for the ills of the horse, or fix the problem with the water.
When we are talking about our children--our human resource--our one renewable capital resource--we not only have the responsibility in an ethical sense, but we also ignore it at our own peril.
Posted by: Margo/Mom | December 11, 2008 9:27 AM
Could someone help me with this? I'm beginning to wonder if our national education debate is based on invalid data. In many states, the state test and other standardized tests have no consequences for the student testtaker. Many students walk in to class on test day and ask, "Does this count toward my grade?" When they hear that it does not, I noticed that many students put their heads down, others try a few questions and quit. Out come the Ipods and phones. Doesn't this behavior make the test results invalid? If yes, why are we having a discussion about gaps if the students aren't even trying? Why are schools closed,teachers moved,superintendents fired? What other institution behaves this way?
Posted by: Bill Mac | December 11, 2008 10:36 AM
Bill:
I don't know how many states/classrooms you have observed testing in, but various states, districts, schools have done various things to "assign stakes," to students. In some states there is an "exit" exam that students must pass in order to get a diploma. Other states rely on end of course exams--again, tied to the diploma. Some states require that the state test count for a percentage of the final grade. I think DC is playing around with paying kids if they "pass," that is score above proficiency.
Individual schools have all manner of inducements--elementary principals who get a pie in the face, or spend the day on the roof reading, if their kids all pass, etc. There are all kinds of test incentives--not that I have ever heard anyone suggest that they are skewing scores upward.
I cannot imagine a testing situation in which iPods and cell phones are allowed, as this violates testing security, invites cheating, and pretty much puts all the school's scores in jeopardy, along with the teaching licenses of any teachers who allow that to happen.
Putting aside my ongoing skepticism about the actual numbers of schools closed, superintendents fired and teachers moved in response to low scores (as opposed to other stuff--like low enrollment, changing district needs, etc), there are in fact various student-level consequences to low passage rates--aside from the obvious ones that relate to the benefits of an education. Identification for interventions programs (ie: summer school, grade repitition), is one. Report to parents is another.
Basically, I would suggest that if students are so un-engaged with education as to substitute the behavior you describe for involvement in taking the test, perhaps there really is a problem in the learning environment of the school that needs to be addressed--regardless of the scores.
Posted by: Margo/Mom | December 11, 2008 11:02 AM
There are no statistically significant declines in these gaps in 4th or 8th grade reading or math. In many cases - for example, 4th and 8th grade math and 8th grade reading - it's not that the black-white achievement gap is declining, but not by enough to be statistically significant. These gaps are actually growing. Sigh.
It's very difficult for education gimmicks to overwhelm the influence of biology.
There are no quick fixes here.
Posted by: TangoMan | December 11, 2008 6:14 PM
We can all find the NAEP reports and see little change in Chicago scores. But the "lense of (quantitative) social science" can capture light and images outside that band,can't it? Worse than Arne Duncan raising test scores would be getting an Ed Sec on that basis. Been there, done that. 8 years ago. W's Houston schools Superintendent, Rod Paige.
Incredulous
Posted by: Incredulous | December 11, 2008 7:01 PM