eduwonkette_header_515.jpg

Through the lens of social science, eduwonkette takes a serious, if sometimes irreverent, look at some of the most contentious education policy debates. (Find eduwonkette's complete archives prior to Jan. 6, 2008 here.)

« COWAbunga! Post-Convention Edition | Main | Grading skoolboy »

Predicting the Near Future*

question_marks.jpg

Sometime soon, with great fanfare, the New York City Department of Education will release this year’s School Progress Reports. (Word on the street is that schools already know their grades.) The School Progress Reports, for better or worse, are the centerpiece of the NYC accountability system. (skoolboy thinks for worse, but more on that later.)

The DOE has made a number of changes to the Progress Reports for this second iteration, and I think that eduwonkette had something to do with that (as did other critics and analysts outside of the Tweed inner circle.) We can expect to see separate letter grades for the three major dimensions on which the Progress Reports are based: school environment (including attendance, and parent, teacher and student surveys), student performance, and student progress. But the overall format appears to be unchanged: most of the grade is based on student progress on test scores, and such gains are not very reliable from one year to the next. There is, in skoolboy’s opinion, a false sense of precision conveyed by these letter grades, as they are based on components that are measured with error, but that measurement error is not reflected in how the grades are calculated. And I’m particularly annoyed at the misuse of social surveys for accountability purposes.

Nevertheless, the DOE is marching onward, and we’ll have this year’s grades to pore over in the near future. (And you can bet that eduwonkette will put on the green eyeshade for this, even though it clashes with her cape and mask.) How many schools will improve their grade from last year to this year? How many will fall? It’s time to make some predictions. What do you think, readers?

Here's a five-by-five table designed to show how this year’s grades are associated with last year’s grade. Each column represents last year’s grade, and each row represents a possible outcome for this year. The column percentages will add up to 100%. Try to fill in the blanks: What percentage of the schools that received A’s last year will receive an A this year? What percentage of A’s will decline to B’s? What fraction will fall further to C’s, D’s, and F’s? At the other end of the spectrum, what percentage of last year’s F’s will remain F’s? What percentage will climb out of the cellar to obtain a D? Will any make the leap from F to A?

crosstab.JPG

As a reminder, last year, about 23% of schools received an A; 38% received a B; 26% received a C; 8% received a D; and 4% (i.e., 53 schools) received an F.

A caveat: The DOE knows that the legitimacy of the School Progress Reports depends on the grades not being too volatile from year to year. If 75% of last year’s A’s became F’s this year, no one would take this scheme seriously. (And if schools that everyone views as exemplary or high-performing got middling grades, this too would call the scheme’s legitimacy into question. So don't expect Stuyvesant High School to get a C.) There may not be very much fluctuation from last year to this. You can be sure that the DOE has constructed this year’s scores so that there’s not too much instability from last year to this year.

But since we believe in incentives on this blog, the reader who comes closest to the actual association between last year and this year shall receive a prize to be selected by eduwonkette—and we know how creative she can be. Be sure to fill in all 25 blanks.

*Employees of Tweed Courthouse, KPMG Consulting, and the Parthenon Group are ineligible for this contest.

TrackBack

TrackBack URL for this entry:
http://blogs.edweek.org/cgi-bin/mt-tb.cgi/4855.

Comments

Great contest! But I'm going to take a pass on this. As the great futurologist Yogi Berra used to say, it's very hard to make good predictions, especially about the future.

Hey Sherman,

As an historian, you probably know that it's not even that easy to predict the past.

This seems like a fascinating probability exercise! I will have to settle for a guess:

80 10 10 5 0
15 75 20 5 5
5 10 55 40 20
0 3 10 30 25
0 2 5 20 50

I hope I am not the only contestant, and that others have emailed their entries! I did have a vague rationale behind my guess, but would not be surprised if I ended up way off. P.S. 8 got an F--under those conditions, who can make a prediction?

Diana,
You are the only contestant. This suggests either (a) vague incentives aren't meaningful enough to the targets of policy to evoke the desired behavior, (b) it's too depressing for followers of NYC education to think about these nasty school progress reports, (c) the mathematical demands of the task were too great, or (d) readers think they have a better chance of winning the state lottery than predicting the distribution of school progress reports.

75 15 15 10 5
10 70 15 10 10
10 10 40 30 15
5 5 20 30 40
0 0 10 20 30

My system based on *blind guesswork.*

My system based on *blind guesswork.*

Jen-M: So's the DOE's! (Not really--it just seems that way sometimes.)

Post a comment

Ground Rules for Posting
We encourage lively debate, but please, no profanity or personal attacks. By commenting, you are agreeing to abide by our user agreement.

The opinions expressed in eduwonkette are strictly those of the author and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Get RSS

Get eduwonkette delivered by e-mail. Enter your e-mail here:

Delivered by FeedBurner

Advertisement
Powered by
Movable Type 3.34

EW Archive