Recently in Accountability Category

Dear Diane,

The bad news does seem to overwhelm the good news of late.

For starters, when talking about education, we need a new vocabulary. The term "achievement" has become synonymous not with the intellectual tasks of schooling, the "using one's mind well" as Ted Sizer put it, but with whatever is measured by multiple- choice tests. The more answers right the better—of course—but otherwise there's little of intellectual quality being measured.

If it's "using one's mind well" that should be at stake, then one might imagine we need tests that rest on demonstrations of how students use their minds! That should be at the heart of any "high-stakes" assessment. And if we could get over our aversion to sampling, even standardized tests could in fact also help us gain understanding of quite complex matters.

One of our readers suggested that David Leonhardt's essay on health care in last weekend's New York Times Magazine was evidence on behalf of measurement. Indeed, what you and I are—I think—arguing against is measuring the wrong things, in the wrong ways, and in the process corrupting the ends we have in mind.

The story of the work of Dr. Brent James, as Leonhardt describes it, is not a refutation, but a support for good assessment. It is a call for assessments that start from the bottom up. And, while the health field has one enormous advantage—little disagreement about what constitutes improved health, except at the two ends of life—it's not easy to gather the right data. With educational assessment, we are stuck with the fact that different "schools of thought" actually exist about means and ends. For example, in my own field and yours, the way we examine the past too often betrays our "deterministic" view of human history. The "causes" of World War I are taught as set knowledge, suggesting that another history couldn't have happened. This mindset undermines democracy and civic participation. Wouldn't it be grand if students were asked "how otherwise" the story of WWI could have played out? And, why not? One of the "habits of mind" that was central to the schools I founded was precisely that: "Supposing that, what if?". But this is a controversial viewpoint—should federal policy settle this difference in outlook?

It takes deep knowledge to struggle through the historical data with such ends in mind, but it is precisely the kind of mindset that history in a democracy needs. Or, so I would argue. It is the mindset that enlivens the debate over the history being written in our own lives. And there is no age too young to start such "imaginary" thinking, based on sound factual knowledge. In fact, playfulness is not just the task of early childhood, but the task of every citizen who cares about the future. Schools rarely do much to keep such "playfulness" alive, but democracy depends upon it.

I've just returned from New Orleans, and a wonderful three days of both celebrating the ideas of Ted Sizer and considering where next to take them. Between sessions I went from my luxury hotel in the French Quarter out to the lower Ninth Ward—that part of New Orleans that still looks like the disaster just happened. We were visiting a school in the Ninth Ward that reopened thanks to the strong will of the principal, her staff, and the school community. But the difficulties in New Orleans made me wonder if anyone is responsible for that old-fashioned idea: the common good.

President Obama's message throughout the campaign was a reiteration of the concept of the "common good," at a time when we were experiencing the impact of a society built around "the more I get, the better." But I agree with you, Diane. What we're hearing now is rather different than what we expected. Neither Education Secretary Arne Duncan's failure in Chicago, nor the mayor's in N.Y.C., nor Michele Rhee's in Washington, D.C., are on the table; it's as though we can sweep that inconvenient evidence under the rug by a constant repetition of good PR.

Producing schools that do right by all children AND by the nation is, in fact, more complex than rocket science. It means, for example, concerning ourselves with the children in that other ward across the levees, not just our own. Democracy works most easily when our personal short-term interests and the interests of the least advantaged match. But democracy can't depend on that.

If Iraq and Afghanistan are about the struggle for democracy, than why can't we see our schools as just as important in this struggle? What we spend in a week or month in Iraq/Afghanistan would allow us to lower class sizes nationwide to match what the most powerful citizens demand for their own children.

So where do we go with this issue, Diane? We need policies that don't make it too easy for the most advantaged to do right for their kids at the expense of those least advantaged. That means, to begin with, defining achievement as something more, not less, ambitious than test scores. The rich can survive that, perhaps, but the nation can't.

Deborah

Posted by Deborah Meier at 12:25 PM | Permalink | No Comments | TrackBacks (0)

Dear Diane,

The absurdities you describe are on the mark and ought to kill the idea of paying teachers based on their students' test scores. But we both know the idea won't die that easily. Even the most renowned of testing experts argue that we're nowhere near being able to produce tests that can do the job of pay-by-score that folks want. I do wonder at times what "they" think they are doing?

The trouble is that when I start down that path I see conspiracies everywhere—for example, that these schemes justify hiring inexperienced and low-paid teachers—who can do scripted test-prep as well as the next guy. It has the handy side effect of destroying solidarity which from a businessman's perspective (perhaps) is a good thing, and keeps teachers away from controversial subjects—and tightly aligned to the stuff being tested. It probably weakens unions. And finally, it paves the way for a marketplace system of schooling instead of a public one (which is then relabeled a bureaucratic monopolistic model). Of course, the latter can be true—and as you know I was an early champion of choice and increased school autonomy for just that reason—within the public sector.

The charter schools have also become I fear another name for vouchers. Operated by private chains with public funding, they offer a kind of distorted marketplace, controlled by test scores standing in for profits. Thus, they kill two birds with one stone: public education and human judgment.

A test of intellectual rigor always in part rests on judgment. Democracy is built on that shaky foundation of trusting our fallible judgments. The central purpose of K-12 schooling in a democracy is thus the training of human judgment. The young can only learn how-to in the company of adults who are doing it. The means and ends are one.

But where you and I may disagree re. the Obama agenda is on the question of nationwide "standards"—e.g. curriculum. I think we are in agreement that NAEP—the only existing national exam—should remain separate and untainted by high stakes. A better labeling/benchmark system—or even doing away with any labels—would be wiser still. We can then focus on in-depth analysis of a wide range of interesting data. Those who want a 50-state common core grade-by-grade curriculum won't be satisfied with sampled NAEP scores because they want to make state-by-state, school-by-school, student-by-student comparisons. They see the competition for test scores as healthy. So, where do you stand?

In that regard, I found the recent Brookings Institution research by Grover Whitehurst et al a helpful warning. He argues that "the creation of common standards will have little impact on our future in a system in which there are also aligned assessments, and aligned curriculum, educators, and accountability for students, and aligned professional development...." And on and on. "Faith," he concludes, is not enough, and a closer look at our most successful "competitors" internationally demonstrates that this is not the path they have taken.

I don't recall the exact data, but someone once noted that teachers make dozens (or hundreds?) of decisions a minute. More than almost any other field (e.g. doctors), partly because they are simultaneously educating 20 to 30 different kids every minute. That's why they need eyes in the back of their heads and a trained instinct about those peculiar silences, mutterings, etc., which only experience teaches us to notice. Kids are complex—and each one is different. The subject matter is hopefully difficult, too. Thus, as Ted Sizer reminded me, be sure and simplify everything else you can, or you'll find yourself simplifying the kids and the subject matter. We've done the opposite under the new "deform" agenda: as school people follow ever more complicated analysis of statistical data! Becoming themselves more like machines. Truly! I sometimes wonder how many hours it would take to accomplish five hours of school time. If we each were teaching one child—home-schooling, in short. Simplifying complex people and information consumes us. It's not all wasted time, but.....

Once in a while, I'd try to make a quick round of my classroom to assess who was catching on to what. The trouble was that I usually got so intrigued by the first child I sat next to that I didn't get to the second. I exaggerate, a little.

Finding the right metaphors and analogies that work for each child is in part an art that takes time to accumulate into wisdom. It takes wariness, too, and sometimes only a good observer can catch our mistakes. It's hard not to hear our own meaning rather than the one the kids are making. Besides, many kids know how to fool us by "looking" smart.

I found this as true in my relationships with adults when I was a principal or head teacher. Because once again we don't all respond to criticism or close questioning the same way and can open up or close down very quickly. Sometimes quite politely.

I wish I could convince the new reformers—those who truly want a better way—that we need to improve and deepen NAEP while school-by-school we develop better tools for assessing that are simultaneously better tools for teaching.

Our final exercises at CPE and Mission Hill and Urban Academy and Beacon and on and on are the culmination of years of teaching/assessing approaches. At culminating years, the staff pull together to show off more formally before real-live audiences, who are assigned the task of "judging." They are, as I say over and over, real "road tests"; what people encounter in the real world of college or the workplace, and they simultaneously respond to the skills and knowledge that we need as citizens who are not easy to fool—at least not often. They serve, in short, democracy first and foremost.

Deb

Posted by Deborah Meier at 10:51 AM | Permalink | 25 Comments | TrackBacks (0)

Alternate title: What Does the Best and Wisest Parent Want?

Dear Deborah,

I am glad to see that you are trying to draw us back to the issues where we have genuine differences! You and I agree that testing and accountability—as currently practiced under NCLB—have become enemies of good education. We would probably disagree on the value of testing. I do think that testing, when sensibly deployed, is valuable. I am not part of any anti-testing movement. I think it is important to know how students are doing, as compared with their past performance and as compared with others in their grade or age group, and even as compared with their peers in other nations. I also see the value of testing for admissions purposes, as there would be no point in bringing a poorly prepared student into a highly selective institution; he or she would certainly fail. The biggest problem today is not testing, but the misuse and abuse of testing, the way that schools, districts, states, and now the federal government are misusing the results of tests to make high-stakes decisions about teachers, students, principals, and schools.

The list that you draw up to describe the kind of curriculum for which schools should be accountable reminds me of the recently released core curriculum standards developed in English Language Arts and mathematics by the National Governors Association and the Council of Chief State School Officers. Like you, they would be comfortable with non-specific outcomes, though they do not make recommendations for history, science, the arts, and other subjects.

We could get into a heated argument about literature, history, and the arts. I don't agree that the value of these subjects and of what is taught must await empirical evidence. While it is not true that "everyone should" read Dante, Shakespeare, Milton, and Homer, and it is likely the case that only a tiny proportion of the population has done so, I don't think that this far-out reference should be used to bludgeon anyone who would argue for specific curriculum content.

I don't agree that knowing history leads any to repeat and relive old enmities. Most old enmities are based on ignorance, fear, and misinformation. The more we know of history, the better able we are to defuse ignorance, fear, and misinformation. Nor do I agree, as you suggest (I hope playfully), that America's success "correlates with its disdain for history."

As for science, I suspect that superstition and belief in supernatural phenomena and conspiracy theories is the result of poor education in science. To the extent that we can teach students to seek evidence and rational explanations, we will reduce magical thinking and encourage the application of reason and intelligence.

For better or worse, I will always side with those who favor more education, not less. We have words for those who don't know history, who have not read worthy literature, who know no science, and who are indifferent to the arts (and think that their doodles while daydreaming belong in a museum): Ignorant. Those who have no basis for reaching an educated judgment rely on hearsay, biases, prejudices, and the opinions that they picked up from their family and friends.

Thomas Jefferson said it first: Those who expect to remain both ignorant and free in a state of civilization expect what never was and never will be.

The Founding Fathers were themselves highly educated. They knew the muck of ignorance that had produced war after war in Europe, unending conflict over religion, and bitter enmities. They wanted something better in the new nation they were creating. To read their writings is to see how widely they read in history and literature, and how much they cared about learning the principles of science. To read them is to raise questions about whether we are as well educated today as they were. Some are, most aren't. As Neil Postman titled one of his books, we are amusing ourselves to death and allowing popular culture—often degraded and corrupting—to determine the content of our minds.

Jefferson would be appalled. I think John Dewey would be, as well. It was Dewey who wrote that what the best and wisest parent wants for his child is what we should want for all children. I expect that Jefferson, Dewey, and that "best and wisest" parent could find common ground were they to break bread together. I would like to be there when they do!

Diane

P.S.: Please let's turn to Arne Duncan's latest statement about his plans for the renewal of No Child Left Behind!

Posted by Diane Ravitch at 9:17 AM | Permalink | 50 Comments | TrackBacks (0)

Dear Deborah,

Over the past week, you and I have each weighed in on the defects of testing. You have been arguing for many years that standardized testing is replete with flaws. I have only recently recognized the ways in which pressure to raise scores, mainly prompted by NCLB, has corrupted testing and accountability.

Our policymakers have fallen in love with the idea that incentives and sanctions can "drive" educational improvement. They believe that if we promise rewards when test scores go up, we will see test scores go up. So they commit hundreds of millions of dollars to give "merit pay" or "performance pay" to teachers and principals, even to students—if the scores rise. Simultaneously, they threaten to inflict serious sanctions on those schools, principals, and teachers if their students' test scores do not go up. They don't dock their pay, but do something worse: They threaten to close their schools, fire the staff, and tarnish the reputation of anyone who taught there.

Behind these promises and threats lies a simple theory: scores are not high enough, because teachers are either lazy, don't work hard, or aren't motivated enough to do a good job. So teachers will work harder and be more successful if they can get more money, and they will work harder and be more successful if their livelihoods and reputations are on the line.

The problem with the incentives and sanctions approach is that it works. It does produce higher scores. We see scores going up in many states, sometimes at rates that defy belief. Some states may actually reach that dreamy goal of 100 percent "proficiency" by 2014.

So what's the problem? The problem is that schools, principals, teachers, and students will reach the goal by hook or by crook. Some states, like New York and Illinois, will play statistical games (like dropping the cut point, or creating conversion tables to change low scores into high scores). Some states will dumb down their tests, carefully field-testing the tests and removing any questions that are too difficult. Some districts will scrub their scores to remove low-scoring students (yes, this has happened). Some districts will find other ways to exclude the low-scoring students, such as by giving accommodations to students who are not usually entitled to them. Some schools will reclassify students to put their lowest scoring students in a group that doesn't "count" because its numbers are so small. Some will even cheat.

Ultimately, we will have what I call the NCLB Paradox, wherein scores go up, but actual educational improvement does not occur. We will see districts where the reading and math scores are through the roof, and where graduation rates have climbed, but where the rate of college-ready students is unchanged. I expect there are many such districts. The one I know best is New York City, which won the Broad award in 2007 for its excellence in improving urban education. Test scores have soared, based on dumbed-down tests; graduation rates are up across the board. Yet when graduates of the New York City public school system enter the community colleges of New York City, 74 percent of them require remediation in basic skills! These are students who passed five state Regents examinations, yet they need to be remediated in reading, writing, and mathematics! This suggests, does it not, that there is something amiss with those impressive test scores and graduation rates?

This raises the question: With scores so often rigged and fraudulent, how can we use them to pay bonuses or to close schools? New York City's last round of phony test scores (noticed as phony even by the august New York Times) triggered a payout of $33 million in bonuses to teachers; the union is laughing all the way to the bank! So millions are awarded in fraudulent bonuses at the same time that school budgets are cut to the bone. Is this the way that big business operates? If so, it is no wonder that we had a financial meltdown.

I fear that American education has now entered into a twilight zone, where nothing is what it appears to be, where numbers are meaningless, where public relations and spin take the place of honest reporting, where fraud is called progress.

Diane

Posted by Diane Ravitch at 8:02 AM | Permalink | 38 Comments | TrackBacks (0)

Editor's note: See author's "P.S." added since entry was first published.

I will have to delay a bit before I can get to the book you recommended. When I finished "Daniel Deronda," I immediately plunged into Robert Caro's wonderful biography of Robert Moses, "The Power Broker," which I am enjoying very much. It is fascinating from the start as a description of the life and times of a man who did so much to redesign New York City, who was celebrated and powerful, but in the end...was the subject of a very unflattering biography by a master historian. The book gives perspective to some of the events that you describe. In the end, people in public life are judged by their real accomplishments and their integrity, not by their wealth, their power, or their press releases. The latter fade, and eventually the record speaks for itself. This is one of the reasons that I love to write history and read history, as over time phony laurels disintegrate, and the applause that is generated by flacks disappears.

I would like to engage you in one of your favorite issues, which is the use and abuse of tests. Over the past few months, as I was finishing my book, I became aware of the startling extent to which the New York State Education Department has manipulated the state test results. So, while politicians crow about their "success" in raising test scores (as if they had anything to do with students' learning!), it turns out that the tests have been rigged in recent years to produce higher scores. The more I learned, the more I wondered if New York was on its way to becoming a national laughing-stock.

I wrote two articles about this. The first one appeared in the New York Post in August under the headline "Toughen the Tests." The article actually was NOT a call to toughen the tests, but a call to tell the truth. I wrote it to alert our new state education commissioner, David Steiner, who assumes his position on Oct. 1, to the scandalous manipulation of test scores by the agency he will lead.

The second article was published by the New York Daily News ("Bloomberg's Bogus Report Cards Destroy Real Progress"). There I discussed the Bloomberg administration's zany school report cards, which this year awarded an A to 84 percent of the city's elementary and middle schools, and a B to 13 percent more. In other words, 97 percent got an A or a B, including seven of the schools that the state says are "persistently dangerous." Only two of 1,058 schools scored an F. The administration thinks this is progress, but even its most ardent supporters on the editorial boards of the New York Post and the New York Daily News complained about rampant grade inflation.

Oh, by the way, the school that saw the biggest drop in its overall score was the Harlem Promise Academy Charter School, the school that David Brooks of The New York Times held up as a national model, claiming that it had closed the achievement gap. Our blog had quite a lively exchange of letters about that school last spring. Seems it dropped from an A to a B; in the present regime of inflated scores, a B in New York City today is nothing to brag about.

I wrote these articles to draw attention to the games that the state is playing with test scores. From 2006, when the state started testing grades 3-8, to the present, the proportion of points that a student needs to advance to a higher level has steadily fallen in many grades. One of our faithful readers, Diana Senechal, conducted an experiment for gothamschools.org, in which she took two of the middle school tests and answered the questions at random; she "earned" enough points to advance to level 2. The number of students who are level 1 (the lowest) has dropped precipitously in these past three years; some very low-performing schools have few or no students in that category, not because instruction has improved, but because the state dropped the bar. The public doesn't know this.

Over these past three years, the proportion of students who are allegedly "proficient" (level 3) leapt from 29 percent to 63 percent in Buffalo, from 30 percent to 58 percent in Syracuse, and from 57 percent to 82 percent in New York City. In 2006, a student had to earn 60 percent of the points on the state tests in math to be proficient; by 2009, the student needed to earn only 50 percent. The public does not know that the bar has been quietly lowered.

The reporters at the New York Daily News have diligently exposed the corruption of state testing by New York's education department. For reasons that I cannot fathom, the reporters at the New York Times have completely ignored the story and continue to refer to the state scores as though they have real meaning. Perhaps the Times will take notice later this year when the NAEP results come out and the public realizes that the claims of double-digit gains are phony. Unfortunately, what the Times does and does not report matters, as some people will believe nothing unless they read it there.**

In June, the Civic Committee of the Commercial Club of Chicago issued a report showing that the score gains in that city (which President Obama cited when he nominated Arne Duncan as secretary of education) were mostly a result of the state's decision to lower the cut scores on the state tests.

When states play games with cut scores and conversions from raw scores to scale scores, testing becomes a mighty scam. As Secretary Duncan said when he spoke at the National Press Club last May, we are lying to our children when we give them a false picture of their progress. When district officials know that the scores are manipulated, yet report their "gains" with a straight face, they become complicit in these lies. When public officials boast about score gains knowing that the scores are the result of game-playing, they too are complicit.

Have testing and accountability become a massive fraud against our children? Do they now serve adult interests while ignoring, indeed disregarding, the needs of children?

Diane

**P.S. The New York Times Faces Facts about State Tests: In my blog entry that was posted today (and written a few days ago), I complained that the New York Times had failed to acknowledge the dumbing-down of New York's tests. Yesterday, Sept. 14, the Times ran an article by Javier Hernandez on that very subject ("Botched Most Answers on New York State Math Test? You Still Pass"). The article pointed out that a student could pass the 7th grade math test by getting only 44 percent of the questions right, and that the passing mark had dropped so far that students could actually pass by random guessing.

Officials at the state Education Department told the Times, apparently with a straight face, that they dropped the passing mark because they made the test "harder." They did not explain why passing rates had soared if indeed the test was harder and comparable to previous years. Apparently the officials think that the rest of us are fools.

So the Times at last has weighed in, but has not yet given the full picture of the extent to which intensive test prep—using clone items—has corrupted the state tests, not only in math, but in ELA as well.

Posted by Diane Ravitch at 8:37 AM | Permalink | 7 Comments | TrackBacks (0)