« On the Art of Listening to Each Other | Main | We're Relying on an Absurd Definition of Achievement »

The Secret of Success and High Test Scores

| 7 Comments

Editor's note: See author's "P.S." added since entry was first published.

I will have to delay a bit before I can get to the book you recommended. When I finished "Daniel Deronda," I immediately plunged into Robert Caro's wonderful biography of Robert Moses, "The Power Broker," which I am enjoying very much. It is fascinating from the start as a description of the life and times of a man who did so much to redesign New York City, who was celebrated and powerful, but in the end...was the subject of a very unflattering biography by a master historian. The book gives perspective to some of the events that you describe. In the end, people in public life are judged by their real accomplishments and their integrity, not by their wealth, their power, or their press releases. The latter fade, and eventually the record speaks for itself. This is one of the reasons that I love to write history and read history, as over time phony laurels disintegrate, and the applause that is generated by flacks disappears.

I would like to engage you in one of your favorite issues, which is the use and abuse of tests. Over the past few months, as I was finishing my book, I became aware of the startling extent to which the New York State Education Department has manipulated the state test results. So, while politicians crow about their "success" in raising test scores (as if they had anything to do with students' learning!), it turns out that the tests have been rigged in recent years to produce higher scores. The more I learned, the more I wondered if New York was on its way to becoming a national laughing-stock.

I wrote two articles about this. The first one appeared in the New York Post in August under the headline "Toughen the Tests." The article actually was NOT a call to toughen the tests, but a call to tell the truth. I wrote it to alert our new state education commissioner, David Steiner, who assumes his position on Oct. 1, to the scandalous manipulation of test scores by the agency he will lead.

The second article was published by the New York Daily News ("Bloomberg's Bogus Report Cards Destroy Real Progress"). There I discussed the Bloomberg administration's zany school report cards, which this year awarded an A to 84 percent of the city's elementary and middle schools, and a B to 13 percent more. In other words, 97 percent got an A or a B, including seven of the schools that the state says are "persistently dangerous." Only two of 1,058 schools scored an F. The administration thinks this is progress, but even its most ardent supporters on the editorial boards of the New York Post and the New York Daily News complained about rampant grade inflation.

Oh, by the way, the school that saw the biggest drop in its overall score was the Harlem Promise Academy Charter School, the school that David Brooks of The New York Times held up as a national model, claiming that it had closed the achievement gap. Our blog had quite a lively exchange of letters about that school last spring. Seems it dropped from an A to a B; in the present regime of inflated scores, a B in New York City today is nothing to brag about.

I wrote these articles to draw attention to the games that the state is playing with test scores. From 2006, when the state started testing grades 3-8, to the present, the proportion of points that a student needs to advance to a higher level has steadily fallen in many grades. One of our faithful readers, Diana Senechal, conducted an experiment for gothamschools.org, in which she took two of the middle school tests and answered the questions at random; she "earned" enough points to advance to level 2. The number of students who are level 1 (the lowest) has dropped precipitously in these past three years; some very low-performing schools have few or no students in that category, not because instruction has improved, but because the state dropped the bar. The public doesn't know this.

Over these past three years, the proportion of students who are allegedly "proficient" (level 3) leapt from 29 percent to 63 percent in Buffalo, from 30 percent to 58 percent in Syracuse, and from 57 percent to 82 percent in New York City. In 2006, a student had to earn 60 percent of the points on the state tests in math to be proficient; by 2009, the student needed to earn only 50 percent. The public does not know that the bar has been quietly lowered.

The reporters at the New York Daily News have diligently exposed the corruption of state testing by New York's education department. For reasons that I cannot fathom, the reporters at the New York Times have completely ignored the story and continue to refer to the state scores as though they have real meaning. Perhaps the Times will take notice later this year when the NAEP results come out and the public realizes that the claims of double-digit gains are phony. Unfortunately, what the Times does and does not report matters, as some people will believe nothing unless they read it there.**

In June, the Civic Committee of the Commercial Club of Chicago issued a report showing that the score gains in that city (which President Obama cited when he nominated Arne Duncan as secretary of education) were mostly a result of the state's decision to lower the cut scores on the state tests.

When states play games with cut scores and conversions from raw scores to scale scores, testing becomes a mighty scam. As Secretary Duncan said when he spoke at the National Press Club last May, we are lying to our children when we give them a false picture of their progress. When district officials know that the scores are manipulated, yet report their "gains" with a straight face, they become complicit in these lies. When public officials boast about score gains knowing that the scores are the result of game-playing, they too are complicit.

Have testing and accountability become a massive fraud against our children? Do they now serve adult interests while ignoring, indeed disregarding, the needs of children?

Diane

**P.S. The New York Times Faces Facts about State Tests: In my blog entry that was posted today (and written a few days ago), I complained that the New York Times had failed to acknowledge the dumbing-down of New York's tests. Yesterday, Sept. 14, the Times ran an article by Javier Hernandez on that very subject ("Botched Most Answers on New York State Math Test? You Still Pass"). The article pointed out that a student could pass the 7th grade math test by getting only 44 percent of the questions right, and that the passing mark had dropped so far that students could actually pass by random guessing.

Officials at the state Education Department told the Times, apparently with a straight face, that they dropped the passing mark because they made the test "harder." They did not explain why passing rates had soared if indeed the test was harder and comparable to previous years. Apparently the officials think that the rest of us are fools.

So the Times at last has weighed in, but has not yet given the full picture of the extent to which intensive test prep—using clone items—has corrupted the state tests, not only in math, but in ELA as well.

7 Comments

Yes, testing has become a massive fraud against our children. It's been obvious to me for some time. What I don't understand is why the major newspapers are ignoring it. Also, why are the universities using these scores as the basis for research?

Shock and surprise! New Yorkers gaming the game! We in the Midwest were just caught stunned by this revelation. (Not that we don't have issues; we're just not as good or blatant at gaming).

So to solutions. As an engineer, I am not trained with the luxury of merely criticizing and drawing a check. Solutions are the challenge and task.

What we know from the Declaration onward is that entrenched governments use their powers to propagate themselves. Like the laws of gravity and relativity, this is a given, something we have to work with or around. Either we limit government's power in something, or we set up mechanisms where that branch of government is put to the people to be tossed out.

Who then do we limit or toss out in the case of education? Once it was the public school boards. Yet in many cities, those worthies became either powerless or complicit in the entrenchment.

And what of the leaders of teachers? For most of the 80s and 90's, principals lost such power along with their local boards. Power went instead to people like Barbara Bullock.

Its rather amazing how little national coverage was given to Ms. Bullock. In the heart of the nation's most troubled school district--and most visible, being our national Capital-- Washington Teachers Union President Barbara A Bullock dropped tens of thousands of dollars a day in casual shopping, spending teachers' hard-earned pay on silver, jewelry, china, crystal. In all, she and two associate leaders embezzled over $5 million dollars from teachers and taxpayers.

The law, alas, does not allow for the true punishment Ms. Bullock deserves. For those dollars should have gone to educating the most desperately in need among our citizens. More, the attention and time she invested in the cover-up and the shopping...were hours which belonged to those children.

Several thousand could have had laptops. She had china and silver. They might have had her best efforts on improving conditions. She was decorating her buffet.


What do New York's tests have to do with embezzled china and silver?

Senator Kennedy's NCLB was always about exposing schools who couldn't make the mark compared with similar circumstanced schools. Nothing more.

So what do I care that NY's proficiency bar has been lowered? I never took the label seriously anyway; it was always arbitrary in every state. Who really knew where to draw the line in the first place?

What we care about is that a measuring tool exists to compare Montrose elementary in Columbus with West Elementary in impoverished McArthur, OH. Or Ms. Smith in the Bronx with Mr. Teeter.

Everything beyond that, the whole "proficient" labeling scheme, always was a canard.


So to the issue of who to toss out and how. We can't just rely on the criminal courts to get rid of louts like Ms. Bullock. Nor can we trust the educrats to determine who is "proficient". We need other tools that directly empower parents to measure their own kids education, to do as they do with butchers and cell phone providers.

To move on when the government product fails the children.


The measuring tool you want for your children is their teacher. For tracking our nation's schools it is the National Assessment of Educational Progress, which was called by Gerald Bracey "the best audit test America has (Lee, 2008)".
Their website says "NAEP provides results on subject-matter achievement, instructional experiences, and school environment for populations of students (e.g., all fourth-graders) and groups within those populations (e.g., female students, Hispanic students). NAEP does not provide scores for individual students or schools, although state NAEP can report results by selected large urban districts. NAEP results are based on representative samples of students at grades 4, 8, and 12 for the main assessments, or samples of students at ages 9, 13, or 17 years for the long-term trend assessments. These grades and ages were chosen because they represent critical junctures in academic achievement.
The Commissioner of Education Statistics, who heads the National Center for Education Statistics in the U.S. Department of Education, is responsible by law for carrying out the NAEP project. The National Assessment Governing Board, appointed by the Secretary of Education but independent of the Department, sets policy for NAEP and is responsible for developing the framework and test specifications that serve as the blueprint for the assessments. The Governing Board is a bipartisan group whose members include governors, state legislators, local and state school officials, educators, business representatives, and members of the general public. Congress created the 26-member Governing Board in 1988. The NAEP assessment operations are carried out with assistance from contractors."

Personally, for the growth of my students, the only tests that helped me were ones that tested things I thought I was teaching, and that gave me immediate information that could allow me to remediate. But for a national scorecard NAEP does the trick, and you cannot narrow your teaching to gain points on NAEP. A truly positive contribution to our public schools would be to reinstate NAEP as the only standardized test, and to free the funds now spent on highly politicized and useless high stakes tests to be used for mentoring teachers identified by qualified administrators through substantial observations of their classrooms.

Diane,

This is my favorite part of your column today: "In the end, people in public life are judged by their real accomplishments and their integrity, not by their wealth, their power, or their press releases. The latter fade, and eventually the record speaks for itself."

If only our education reform were guided by this kind of wisdom.

Diana Senechal

This inflation happens in all 50 states. As districts across the country collect more test scores from the classrooms everything reported up the chain becomes high stakes and therefore inflated and useless for any purpose other than propaganda. Teachers are left with no believable test scores or grades for their incoming students.

The NAEP has been a good reality check that suggests every state is reporting phony data. People need to realize that it too has a built in inflationary creep as more students recieve accommodations on it. A few years ago the feds posted practice tests on the web so teachers could rehearse their students, inflate the scores, and make the President look good. I have no idea how many teachers might have done this, but I'm sad to say that I don't even believe the NAEP anymore.

Ms. Ravitch,

Thank you for adding your voice to the chorus on states playing games with test data.

I recently have had a long (12-week) battle with the folks in our state ed department merely to explain to me how our tests are scored. After phone calls, and dozens of e-mails, and many sets of very reasonable questions, our director of testing will not give up the information. Then, this year, they let kids who didn't pass the test take it again. Voila! Scores shot up dramatically. As to why they allowed retesting this year, no one will take the credit. Everyone simply says that someone recommended it.

There are stories like New York's and North Carolina's (where I live) in every state. Our testing system is a sham. And this, to me, invalidates virtually all arguments based on test scores, standards, and any other quantitative measurements for judging school and/or teacher quality.

We simply can't enter into another era of reform based on false data. But, as you point out, the public doesn't know about this and the press is usually asleep at the wheel.

If we can oppose the death penalty -- and have states stop using it -- because there are too many errors in too many cases, shouldn't high stakes testing be opposed -- and curtailed -- on the same basis?

Steve Peha
www.ttms.org

Not all states water down school tests as a matter of policy, but when they do it's good to have a public voice that carries weight call out the sham. Thanks for doing this, Diane.

The idea of state-wide school tests is not necessarily bad in itself. But when these tests are construed as a check on schools, rather than a check on students, you get all sorts of warped effects.

For one, if teachers are kept out of test grading, it is very hard to publish the test results within a few days, making tests completely ineffective as an educational tool. As a rule, you want to point out mistakes to a student as soon as possible; if students wait months for test results, they will not learn anything from their mistakes.

Second, if the main intent is to test the schools, and the prime goal is an accurate test score, then multiple choice test questions should be in theory all right to use. (That is, if the test makes sure that completely random answers don't guarantee a >50% pass rate, as was the case of the NY State Math test).

But if the interest is to use the test as an educational tool, you don't want multiple choice questions in the test. Let the kids formulate their own response; it'll teach them to express themselves.

There is also something to be said about listing three wrong answers as choices next to the correct one: there is an old education rule, which says you should never admit an incorrect statement written down for students unless it's immediately corrected.

A better way to set up the tests would be to have them around a common state (or nation) wide curriculum, aligned with the text books actually used in class, with open ended question, graded by teachers not by machines, with results made available within days, and with hard consequences for student class/track assignment.

Comments are now closed for this post.

The opinions expressed in Bridging Differences are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Follow This Blog

Advertisement

Most Viewed on Education Week

Categories

Archives

Recent Comments

  • hertfordshire security installers: Greetings. Great content. Have you got an rss I could read more
  • http://blog.outsystems.com/aboutagility/2009/04/challenges-of-scoping-and-sizing-agile-projects.html: I would like to thank you for the efforts you've read more
  • http://acousticwood.net/mash/2008/03/yeah_off_to_the_uk.html: Between me and my husband we've owned more MP3 players read more
  • buy cheap metin2 yang: When you play the game, you really think you equipment read more
  • Nev: Anne Clark - If a Dr. instructs a patient that read more