« It All Depends On... | Main | Scouring for Racial and Cultural Bias in Tests »

Right Answers, Wrong Answers

| 13 Comments

Dear Deborah,

I didn’t suggest that “good medical care” was “test-prepping.” Just goes to show how easily words are misunderstood, how important it is to teach grammar, syntax, spelling, etc. so as to improve the clarity of our expression. When I went to public school in Houston, our English teachers devoted half of every year in their classes to teaching correct grammar. It was never fun, but it was very valuable. I am reminded on a daily basis of the importance of good grammar and syntax; without them, we will all of us have trouble communicating what we mean. And as you well know, your notion of democracy—and mine—depends on citizens being able to reason together, to listen to one another, and to get the drift of discussion because they share a common vocabulary with a considerable body of shared knowledge.

Like you, I too have always been puzzled when newspaper articles announce with alarm that half the children in a grade are “below grade level.” Grade level, as you note, is the norm for a particular grade, and at any given moment, half the students will always be above grade level, while the other half are below. This will be true whether achievement goes up or down.

The trend in recent years, inspired by the National Assessment of Educational Progress, is to devise criterion-referenced tests, where students are judged by whether they reach a certain level of knowledge and skill (basic, proficient, advanced). There is no magical “grade level” in such tests. We regularly see NAEP results where only a small fraction are advanced, and where most students are clustered at basic or below basic.

As for the question of right answers and wrong answers, I agree with you in part and disagree in part. I share your revulsion for test questions that gauge students’ opinions, feelings, and values. When I served on the NAEP board, I frequently reviewed test questions and made sure that the national test avoided such stances. It seemed to me outrageous to grade a student on whether they expressed a politically correct attitude.

Where I disagree is that you seem to suggest that there is no such thing as a “right” answer. If I asked you who was elected president of the United States in 1960, don’t you think there is a right answer? If I asked you to multiply ½ x ¾, don’t you think there is a right answer?

Given my experience with NAEP over seven years, I agree with your concern about the meld among IQ questions, reading questions, and math questions. It is almost inevitable that a reading question or a math question will turn out to be a test of intelligence; it is very hard to separate comprehension from whatever we call IQ. To take the issue one step further, I was troubled to see the trend in which math questions were turned into reading questions, a trend found in most tests in the past 15 years or so.

Part of the difficulty comes from the effort to test critical thinking and problem-solving ability in math. The goal sounds laudable, but the questions tend to be so wordy that a student with good math ability but poor reading skills is handicapped by the form of the question.

By the way, I can’t speak for ETS’s historical practice of playing around with items, as you put it, “to ensure that male scores more properly matched or surpassed female scores.” Whether that is true, I do not know. What I do know is that girls regularly outperform boys in reading on NAEP, by a wide margin, while boys do better than girls in math, but the gap is smaller than in reading.

I think your suggestion that the performance of black students, as compared with white students, is a function of the test questions is simply wrong. For at least the last 20 years, the tests—national and state—have been scoured to remove any hint of racial or cultural bias (forgive me, but I wrote a book about this, "The Language Police"). The racial gap on test scores is large, and I personally attribute it to inequality of educational opportunity, not the wording of the test items. The suggestion that the gap would disappear if the tests were changed doesn’t hold water.

I agree with the hint in your postscript that the achievement gap is in part a function of very large gaps in income and social advantage. That should be no surprise. Demography and socio-economic status have long been powerful determinants of educational achievement. Just because we have a federal law that declares that all children will be proficient by the year 2014 does not change those facts of life. I don’t know of anyone who thinks that the gaps will be closed and that no child will be left behind by 2014.

It seems safe to say, however, that thousands—maybe tens of thousands—of public schools will be declared failures by 2014 for not having met that impossible goal. This will give an enormous boost to the forces that are promoting privatization of public education.

Diane

13 Comments

Diane,

You stated, "The racial gap on test scores is large, and I personally attribute it to inequality of educational opportunity, not the wording of the test items." A number of years ago I would have agreed with you on this statement. However, as a result of education reform, state after state has been taken to court in an attempt to rectify these inequalities. While some states, districts, and individual schools are not there yet completely, the problem has been documented, acknowledged, and is finally being addressed. The exact extent of a disparity still existing in any state must be considered vastly improved over what it was five, ten, or twenty years ago.

I also do not believe the achievement gap is the result of the wording on test items. Talk about a cop out for poor performance. That’s like Willie Randolph attempting to pull the race card for the dismal performance of the New York Mets to this point in the season. Come on Willie! He manages one of the highest payrolls in major league baseball and he knows the Mets should be doing better. He simply has to improve the team’s performance or he’ll be terminated. It has nothing to do with race. Jim Leland in Detroit is in the same predicament. If the Tigers don’t improve soon, his job will also go to someone else.

As a result of education reform, the achievement gap has also been documented, acknowledged (finally), and is now being addressed. To this point, a remedy appears as elusive as it is variable. While there are pockets of success, what seems to work in one situation doesn’t necessarily hold true across the board – sort of an enigma wrapped in a puzzle. However, there are constants that have at least been identified that, years ago, many refused to even acknowledge. Poverty, race, minority cohorts, non-English speaking students, as well as students identified with learning disabilities, have all been identified in discussions about this defined gap.

Additional variables I’d like to see acknowledged in any future discussion on the achievement gap are the lack of parental involvement among failing students as well as the (non)conforming disobedience of some of the failing cultures. For reasons of political correctness, even mention of either of these two issues seems to be taboo. If the emperor has no clothes on, then guess what – HE’S NAKED! Absent the inclusion of these two obvious contributors, officials nationwide are only fooling themselves if they think they’re ever going to eliminate the existing achievement gap in our pluralistic schools.


Paul:

There has been research of the type that you suggest. While I don't have any citations at my fingertips, I believe that any differences by ethnic and socio-economic status are far less profound when looking at the elements of "parent involvement" that have the greatest impact on student achievement. These include such things as homework help and talking with students about school--things that are fairly invisible. Where there are much larger differences (but less evidence of impact) are in the kinds of "parent involvement" that falls into a fairly traditional definition--such things as booster clubs, fund raising, etc.

The Public Education Network has recently published the results of three years of public hearings on No Child Left Behind. Their findings are of interest because NCLB has gone further in codifying some of the parent involvement requirements that have always been present in Title I. Based on their hearings, from parents and communities, parents are typically excluded from any decision-making or policy-making function within schools. This is despite requirements for inclusion within the law. PEN suggests a need for more "teeth" regarding enforcement of specifics regarding the involvement of parents.

As a for instance--does your district have a parental involvement policy? Is it accessible? Is it followed? Is it evaluated? Did parents help to write it?

I was in attendance at the National Math Panel meeting held at Stanford when Jim Milgram (former Stanford Math professor and IES board member)presented strong evidence that twenty percent of the NAEP math questions were either not math questions or were wrong. We have enough problems measuring student performance without overtly dysfunctional tests. Do other countries suffer from this level of inaccuracy in their testing? It reminds me of when Richard Feynman reviewed the math textbooks for California and kept finding errors in the content.

What I do know is that girls regularly outperform boys in reading on NAEP, by a wide margin, while boys do better than girls in math, but the gap is smaller than in reading.

Have you looked at the NAEP reading test? It is overwhelmingly a writing test. Students have to read an essay and then write a response to it.

ACT and SAT tests are far better assessors of reading, although neither is perfect. The ACT has been issued to entire state populations, and shows next to no reading gap between the genders. That's almost certainly more accurate than the NAEP.

The NAEP is a joke. Too few questions, not offered to everyone, and most of its questions test something other than the subject matter they're purported to cover.

Cal,

Yes, I have "looked at" the NAEP reading test. I was a member of the NAEP governing board for seven years and I read every test question on the reading tests. They all require comprehension; some require students to write, but not all of them do. The NAEP tests are not a joke.

Diane Ravitch

From To take the issue one step further, I was troubled to see the trend in which math questions were turned into reading questions, a trend found in most tests in the past 15 years or so.

to

They all require comprehension; some require students to write, but not all of them do.

So it's a bad trend in math (in which some, but not all, of the questions require reading and writing), but it's peachy in reading comp tests to conflate reading and writing? And it's odd that you didn't mention that the reading test does exactly what you complain about in the math test.

I'm certainly unconvinced. The NAEP tests are a joke. I find it difficult to take seriously your declarations about reading comprehension based on a test that requires written answers and essays.

Margo,

"PEN suggests a need for more "teeth" regarding enforcement of specifics regarding the involvement of parents." I'd be all for that. A well defined, measurable policy could go a long way toward alleviating at least some of the more obvious problems teachers face, especially in urban classrooms.

I've often wondered about China's policy of allowing only one child per family. Many argue this is too dictatorial on the part of their federal government. However, some contend it's a socially responsible posture for the most populated country on the planet. The government has clearly defined a measurable policy for parents to follow regarding perhaps the most intimate/private of decisions parents make.

It would be a very slippery slope for a purported democracy such as ours to even consider something along the lines of our schools (government) mandating acceptable/appropriate parental behavior, at least in the area of child rearing. It certainly could not lean toward dictatorial but if it resulted in a more socially responsible group of youngsters, then it must be considered. To continue to avoid the discussion will sadly result in the continued social problems experienced for too many generations of poor and minority families.

To answer your question, my district has parental guidelines (not on involvement). They were developed originally by a local hospital for new mothers to serve as an informal guide on appropriate behaviors for parents in a variety of private and social situations. It obviously cannot be policed but it is available and I honestly believe it has helped countless families.

Paul:

I think that you are embodying exactly the problem that parents are confronting (and citing to PEN). The guidelines that your district uses sound like a one-way list of shoulds, based on the assumptions that parents need to be trained in ways to be "appropriate." The parent involvement requirements of No Child Left Behind call for a collaborative role for parents in their child's education and in school improvement efforts. In support of this role, the act requires that a fair amount of data be made available to parents (typically via the annual school report card--although some specialized information may be disseminated in other ways, such as notification of Highly Qualified Teacher status and School Improvement Status). An annual meeting (for Title I schools) is required to share with parents such things as school improvement goals and strategies (or the school improvement plan). The school is required to involve parents in writing the school's parent involvement policy, and in evaluating the results.

My cynical observation (born out by PEN's research) is that schools are not terribly interested in carrying this out--and in fact may actively build barriers to parent involvement at any meaningful level.

Diane,

It seems possible that when Deborah wrote "The notion of good medical care as 'test prepping' is delightfully bizarre," she was referring back to Rothstein's point, not to yours. I can't be sure, of course, but it seems likely. In any case, your point about clear language holds true.

I enjoy your EdSpeak: A Glossary of Education Terms, Phrases, Buzzwords, and Jargon. Some might take issue with individual definitions--but I don't see how anyone could argue against having such a glossary in the first place. We can debate the meanings of terms, but let use them carefully!

And I agree about the importance of teaching grammar at a deep level. I enjoy browsing through Exposition of the Grammatical Structure of the English Language by John Mulligan, published in 1874 (and now downloadable through Google). He writes: "The word 'but' performs such important and various and apparently dissimilar functions in our language, that it has given occasion to much speculation among grammarians and philologists." Even if some of the information is dated, the insights are not. If the word "but" warrants investigation, then what word does not?

Almost always, after writing something (and posting it, in some instances), I spot an ambiguity. It seems that never ends. With the awareness and the knowledge, though, one may improve over time. That's why schools should bring grammar back, and study literature from a grammatical standpoint as well as other perspectives.

Grammar and meaning are inseparable. When I read Gogol, I see again and again how his wonderfully playful sentence structures contain the essence of his meaning. A Shakespeare sonnet, with its great grammatical density, can teach more about persuasive writing than many a "model" persuasive essay.

And of course I found a bunch of flaws in the above comment. No one is exempt, not even people who love grammar and words.

Paragraph 1, sentences 1 and 2: "probable" or "likely"? It seems flimsy to say one, then the other. "Likely" is the word I would choose.

Paragraph 1, last sentence: "In any case, your point about clear language holds true." I would add "for everyone, myself included."

Paragraph 2, last sentence: "We can debate the meaning of terms..." I think "may" would be better than "can" here, though "can" is not wrong.

Paragraph 2, last sentence: "But let use them carefully!" "Let's," not "let." Eek!

Paragraph 4, sentence 1: "Almost always... I spot an ambiguity." I would add "...or outright error, usually more than one."

Last paragraph, last sentence: "grammatical versatility" might have been preferable to "grammatical density."

All this in a carefully written comment! And there are probably more flaws. All this reinforces the point that the search for precise language does not stop. But if kids do not receive the tools for such a search, they will be caught in vague, imprecise language with its attendant dangers.

I agree with Margo that parents are not typically welcome in school systems, even when mandated to be so. I have witnessed the exclusion of parents numerous times, and often either an overt or passive/agressive hostility toward parents.
I have also been involved with a few parents who are on a teacher crucifixion crusade, which sets up an adversarial relationship and contributes to further perpetuate this negative cycle. All our students suffer when either of these scenarios occur.

Kim:

I have also seen some cases of "activism gone wrong," although my perspective is more conducive of seeing this at the district, rather than teacher, level. I always wonder, when I see some of these folks, what the outcome might have been had their initial forays been greeted with an invitation to be involved in solutions rather than being ignored or brushed aside. It sometimes seems that there is a face-off that starts with the "go away" message that gets stuck in a very unhealthy way. The response of "you can't make me" is sometimes the battleground.

To be fair, also, I have seen teachers in their first or early years of teaching, who are simply unequal to the task of collaborating with parents who may have the advantage on them in years, experience, possibly education. To expect them to go head to head with an unhappy parent unassisted is really unreasonable.

I am interested in your perspectives on what Michigan is now implementing. 4 years of Math in High School for all students. Sound good? Yes. I agree. Now the problems. Algebra 1 must be in High School. It no longer can be in 8th grade. Why? No reason given. How does this help kids. Children can no longer take Calc because they cannot get ahead. In the quest to 'get ahead' they are leaving some children behind. Michigan is demanding Algebra 1 and 2 and Geometry and another year of math in High School. Are all children in America really college bound?

Comments are now closed for this post.

Advertisement

Most Viewed on Education Week

Categories

Archives

Recent Comments