Dear Deborah,
I enjoyed seeing you honored as a hero of education by FairTest last week, which established an annual award named for you. If anyone had told me five years ago that I would be at that event, I would have thought them mad. This is what “Bridging Differences” has done for me, I suppose.
At the event, I was surrounded, not surprisingly, by educators who have long believed that standardized tests are more wrong than right, or that they are a crime against children’s nature, or worse. I continue to believe that we can get valuable information from standardized tests and that they can help with diagnosing problems and needs. The information derived from testing can be useful, but lately I have begun to see how often test scores are being misused to punish kids, teachers, and schools and to mislead the public.
As it happened, New York state just released the results of its annual tests of English language arts and mathematics, and the scores soared across the state to an extent that was literally unbelievable.
The state Education Department released the math scores last week. From 2006, when the current testing regime’s trend line begins, to today, the percentage of kids meeting state standards (that is, scoring a 3 or 4 on a four-level proficiency scale) has gone from 65.8 percent to 86.4 percent. In 8th grade, the proportion meeting or exceeding standards leapt from 54 percent to 80 percent. The gains for black and Hispanic students across the state were huge—for black students, from 45 percent to 75 percent, and for Hispanic students, from 51 percent to 79 percent. White and Asian students are already close to the ceiling, at 92 percent and 95 percent respectively.
Some districts saw increases that defy anyone’s wildest dreams: In Buffalo, the proportion passing flew up from 28.6 percent to 63.3 percent; in Rochester, it went from 33.1 percent to 63.4 percent; in New York City, from 57 percent to 82 percent; and Syracuse nearly doubled from 30 percent to 58 percent. All in four short years! At this rate, everyone will be proficient well before NCLB’s deadline of 2014.
The news media reported the dramatic gains with a straight face. The superintendents of Rochester and Buffalo basked in the limelight. Mayor Bloomberg said the scores proved the value of his one-man control of New York City’s schools, although surely his reign had nothing to do with the even larger gains in other cities in the state. Only the Rochester newspaper asked in an editorial whether these gains made any sense.
Now the New York Daily News has done an analysis of the math tests and concluded that the state tests got progressively easier from 2006 to 2009. Kudos to reporters Meredith Kolodner and Rachel Monahan, who beat The New York Times to this statistical scandal. Kolodner and Monahan had the smarts to turn to Jennifer Jennings of Columbia University, who was formerly the blogger for Education Week known as eduwonkette; Jennings analyzed the tests and discovered that the state has been testing only a fraction of its math standards, and teachers are able to predict which standards will appear on the tests.
Jennings also found that nearly identical questions have appeared every year. “In 2009, at least 14 of the 30 multiple-choice questions on the seventh-grade exam, for example, had appeared in similar form in previous years,” said Jennings. Teachers and principals chimed in and agreed that the questions were predictable and students are taking frequent practice tests that teach them the format.
A teacher explained to me recently that “we drill down into the state test to predict what will be tested,” and then students practice those questions, again and again.
My guess is that if the students in New York state were given a math test from another state—one that they had not been primed for—their scores would be much lower.
U.S. Secretary of Education Arne Duncan keeps telling states and districts that they are “lying” to kids when they tell them that they are doing just fine, but they really are not. Just last week, Duncan said that many states are “lying to children and their parents, because states have dumbed down their standards.” New York is a perfect example of what Duncan means. The proportion of students who pass the tests keeps going higher and higher every year, but when the 2007 NAEP scores were released, the state had flat scores in everything but 4th grade math.
What we see in New York state is institutionalized lying, according to Secretary Duncan’s definition. The state is well on its way to becoming a national laughingstock if it keeps up this Ponzi scheme whose victims are its students.
Mark Twain wrote, “Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: ‘There are three kinds of lies: lies, damned lies, and statistics.’”
The New York State Education Department is showing how easy it is to lie with numbers.
Diane