« Worrying About When the Other Shoe Will Fall | Main | Addendum to NYC Grades Entry »

Bad Grading in NYC

| 7 Comments

Deborah,

You mentioned the new grading system. Our readers may not be aware of what is happening, so let me recapitulate the system as best I can. "Best I can," to be sure, because it is not a transparent system and its calculations are extremely obscure. At bottom, it amounts to this: Each school in the city school district (but not charters) is given a letter grade from A through F. The letter grade is based mainly on the state's standardized test scores. The grade is comprised of both performance (the school's scores) and progress (the school's value-added). Added in are "quality reviews," in which each school is judged by outside reviewers for such things as its environment, as well as surveys completed by parents and teachers. (Editor's note: See Diane's Nov. 14 addendum on the environment component.)

This may not be 100 percent accurate, as the formula is complex. But observers have faulted the system because the underlying tests on which most of the grade is based were not designed for this purpose. Year-to-year fluctuations in scores make such judgments questionable. There is also reason to wonder about the value of giving a school a single letter grade, which stigmatizes some schools and puffs up others. It would be akin to giving a student a single letter grade, and saying in effect, "you are a D student," when it would make far more sense to give grades or evaluative judgments about a student's performance in a variety of subjects, which may be quite variable.

In this new system, which is related to the Florida system of grading schools, some highly reputable schools have received a B or C or even in one case, an F. The high-performing schools are penalized by the grading system because so many of them are near the ceiling and are likely to experience fluctuations down a few points, which condemns them. A much-admired school in Staten Island, for example, received an F, even though its students regularly had the highest scores in its district; but in the last year, the proportion of students who passed the state tests had declined, not by a lot, but nonetheless it was a decline. So this well-respected community institution was graded an F.

Meanwhile, a number of schools that have been identified by the state as failing schools have received an A or B. The Village Voice reported that Stuyvesant High School, one of the competitive schools that is a jewel in the city's crown, initially received a C, but someone talked the higher-ups into raising the grade to an A. I suppose if the grading system had actually awarded a C to Stuyvesant, the grading system would have been laughed out of town. Meanwhile The New York Times reported that Bard Early College High School has protested its C and the president of Bard, Leon Botstein, has made a personal appeal to the chancellor to raise the school's grade to an A.

The best deconstruction of the grading system that I have seen to date is in a marvelous blog called eduwonkette.com. This is an anonymous scholar who is admittedly female, but has otherwise decided, for reasons of prudence, to keep her identity secret. Her analyses of policy decisions have been brilliant, bold, and incisive. Many people are talking about her, because she usually adds a sharp dimension to whatever is in the news. She is wise to remain anonymous. Her bottom line: the methodology used to grade schools is fundamentally inaccurate and invalid.

Parents in some districts are outraged, and educators in outstanding schools that received low grades are demoralized. It is hard to know what the value of this exercise is. The school system's leaders seem to believe that shame and humiliation will incentivize the staff in low-grade schools to do a better job and that this will be sufficient to promote improvement. Of course, the best way for a school to raise its grade will be to focus on the state tests even more than they have done up till now. Some principals will realize that it is time to toss out the arts, physical education, history, and any subject other than reading and math. The only thing that counts is making progress on the state tests.

I don't know if we write too much about New York City, but we have reasons. For one, we both know this system well. For another, New York City is now being touted as a national model after winning the Broad award. Caveat emptor.

Diane

7 Comments

Dear Ms. Ravitch,

I noticed that you stated that the Quality Review (done by outside reviewers) played a part in the grade each school received. Unfortunately, this was not the case. The "environment" grade that comprised 15% of the school's letter grade was based on the parent and teacher learning environment surveys sent home last year, which were not done by "outsiders."

The Quality Review that each school had done last year -- and was done by "outsiders" -- was ignored in the grading formula, which I think is a fundamental problem with the grading process. The Quality Review provided a comprehensive look at all facets of a school (almost like a portfolio assessment), yet the Dept. of Ed. choose to ignore this in their grading process and only look at how students performed on a two-day test. There are eight schools that were deemed "underdeveloped" by the Quality Reviewer -- the lowest rating -- that received A's, and seven schools that were deemed "well developed" -- the highest rating -- that received F's. I'd like to think that most educators find portfolio assessments to be a much better indicator of how someone is doing than how someone does on a high-stakes test. I just wish the New York City Dept. of Education would recognize this.

Good point, Stephen. The lack of correlation noted above is pretty stunning. The parent/staff surveys are interesting data--and might be worth learning more about. Do folks in good schools expect that their responses will count more and thus respond more? Is the percentage of responses significant? Were parents informed about the purpose of the responses? Even though it was a very small factor in the oveall grade, it's an interesting question.

Thanks for the additional input.

Deb

Every statistician and researcher not involved in this has said the approach is unsound: not enough data, too much random change in test scores, etc... From Eduwonkette to the NY Times Editorial Board, everyone knows the letter grades are not reliable. Many people have questioned whether the grades mean anything at all. Countless parents and teachers have weighed in to say anonymously that their own high grade does not make sense. A small set of schools have been able to appeal their scores on the basis that the raw data may be wrong, but no one knows why or how it will end.

So... what? So what? Review the facts.

1. The mayor has mayoral control of the schools. He reports to himself. Legally, who could stop him?

2. The union bargained away any right to appeal the grades when RW agreed to allow the use of the letter grades. It is a little late now for the union to complain, isn't it? Individual schools will have to fight it out alone. Will a school with an "A" throw away the money? Will a school with a "D" try to duck out in shame?

3. From a public relations perspective, those who are complaining openly have been positioned as whiners and loons. What do most people think when they hear that Bard College is appealing: rich whiners didn't get the grade they wanted. Awwww. What do most people think when they read that schools should not get grades because all grading and testing is damaging to self-esteem: Loons. Suck it up. Right?

4. As for more challenging critics, their integrity can be attacked by people on contract. PR experts. "Smear Merchants," Bill O'Reilly calls them. Diane Ravitch, flip-flopper. She doesn't cares about children reading on grade level.

5. The scores can't be checked because the calculation is too complex to explain. Scores and data have been reported, but to the best of my knowledge no independent auditor has checked the basic math working from raw data to final grade. This is a standard procedure for standardized tests. Mayor Bloomberg understands accounting audits. Skipping this step would not be an accidental oversight.

So, what happens?

You can't prove there was an error because you can't run the calculation; most people don't understand the statistical problems with the grading; there is a PR machine working to support this system, the mayor has total control; and the union has already signed off on the grades, surrendering before the first shot was fired.

So, then, what?

I don't know, but I do know that many places across the US are already looking to replicate the mayor's success. The future of education in places many New Yorkers have never even heard of will ride on whether New Yorkers embrace a system that apparently miscalculates educational quality. Wait until people start to teach “lessons learned” from schools inappropriately given "A" or "B." How long until worthwhile programs at schools given "D" or "F" are unfunded and ended? And those "best practices" will be taught at professional development seminars all across the nation.

It is not an exaggeration to say that the nation is counting on New York. The future of our education is in your hands.

Dear Prof. Revitch,
I saw your response to the “City School report cards” article in the NY Times, and agree with you that there should be a grading system based upon different “subjects”. I would simply add another “subject” - that of parents participation in their Childs’ school. James Coleman’s (Sociologist) data has repeatedly suggested that the Catholic schools always do better than their public school counterparts (servicing the same SES population) because of the parent participation.

Steven Gerardi, Ph.D.
Prof. of Sociology
CUNY

Dear Prof. Revitch,
I saw your response to the “City School report cards” article in the NY Times, and agree with you that there should be a grading system based upon different “subjects”. I would simply add another “subject” - that of parents participation in their Childs’ school. James Coleman’s (Sociologist) data has repeatedly suggested that the Catholic schools always do better than their public school counterparts (servicing the same SES population) because of the parent participation.

Steven Gerardi, Ph.D.
Prof. of Sociology
CUNY

Steven, would a parent like me who argues with teachers, once pounded my fist on a table, and has also once walked out of a meeting with teachers get a bad grade?

What happens if the teachers were essentially incompetent, ignorant, and could find effective curriculum or good research if it was handed to them?

Dude, it goes both ways, please buy a clue.

Yes Dickey you and your Child's school would get a good grade! The fact that you are thinking and acting puts you in a class of parents that are few.
Steven

Comments are now closed for this post.

Advertisement

Most Viewed on Education Week

Categories

Archives

Recent Comments