Dear Diane,
You didn’t suggest that good medical care was test-prepping—that was my translation. But it does seem to me that in effect that’s not a bad way of looking at it—most standardized tests rest on experiences that don’t come to us from school, but from the prepping that life has offered us. It may even account for the interesting story I just read—more next week—on a Chicago study that demonstrates that actual test-prepping may hurt scores!
As to testing itself: Yes, reviewing questions for bias and using judgment to determine “grade level” rather than statistics are important. But we disagree about whether, like judgment in general, we can be “neutral”, devoid of politics as well as bias.
Agreed: there are “right and wrong” answers, and you give two examples of such. But I think we both agree that these examples are not at the heart of what we mean by being a “well-educated” person. Therein lies the dilemma. It shows up, we all assume, more on the language arts sections of the test than in disciplines like math and science. Yet one of the first critiques of testing I read was by physicist Banesh Hoffman in “The Tyranny of Testing”. He noted that some answers on the SAT physics test were just plain wrong (although ETS insisted that they were still good “discriminators” between smart and less smart test-takers). Too few, he noted, helped him see who was of a “scientific” mindset. In our field, Diane, asking for the “causes” of WWI, or the Civil War, were an obvious effort to overcome this by asking for “reasoning”, not just “facts.” But in the end they rewarded conventional memory of conventional reasoning, not evidence of historical thinking. I recall that one history question required one to believe (or simply pretend to) that the cause of fascism was the Versailles Treaty. Surely a theory, but a fact?
Scouring for racial and cultural bias—and removing obviously biased items—is good practice. But the bias I’m describing is only possible to detect if one investigates it in the manner I did. What I was “judging” was the reasoning from the evidence that students demonstrated. They occur even on “factual” questions when the facts correlate with certain experiences that children are more or less likely to encounter.
On the NAEP, girls do better in reading; on the SAT, they do not. Who is “right”? Even the decision to rest the SAT on English and math is a decision that has an impact on results. I’m not necessarily condemning this decision, but pointing to the consequences and our ability to read too much into them.
You and I agree in seeing socioeconomic status as a powerful determinant, but logic alone does not get us to this conclusion. Charles Murray builds a powerful counter-argument, which some of our readers find compelling. He argues that, in large measure, the reasons whites and Asians do better than blacks and Native Americans, ditto for Rich vs. Poor, on tests is the same reason they are ahead in the world of wealth and power; it’s due to what’s “inside” their brain. Ditto perhaps for males vs. females?
Surely, beliefs of this sort are not only still around, but they have a long and sustained history in many parts of the world. They have an impact on those who come out ahead and those who come out behind. Living within a culture which takes your “lesser” abilities as the norm, and your signs of intelligence as the exception has a deep and abiding impact. It is one of the reasons I am so opposed to NYC’s intention of testing all 5-year-olds for intelligence, and then (of course) reporting the results to teachers and parents. This is a step that must be stopped because, aside from its many pernicious effects, testing of 5-year-olds is notoriously unreliable as most independent testing experts and others affirm. (More on this very soon; along with the story of 160 out of 162 8th graders in the Bronx who refused to take a practice social studies Regents exam on the grounds that this was further wasting their time.)
Maybe there’s a paranoiac streak in my nature that makes me wonder whether it was sheer ignorance that led the authors of the NCLB legislation to require that all testing gaps be closed by 2014. NCLB was passed as much as a “sentiment” as a real vehicle for improvement. But setting the goal at 2014 was naughty. I think it expressed a coalition of the naïve and the clever. NCLB was hailed by many conservatives as a test of the capacity of public education to do its job properly. Setting an impossible goal has opened the doors for a lot of crisis-reorganization plans, including privatizing schooling. It may, in the end, also reinforce old racial and class prejudices. Surely that wasn’t Ted Kennedy’s idea—and in fact he put his weight behind NCLB precisely to stop worse measures. But in doing so he unleashed something he did not foresee. (He’s on all our minds these days.)
Deborah