eduwonkette_header_515.jpg

Through the lens of social science, eduwonkette takes a serious, if sometimes irreverent, look at some of the most contentious education policy debates. (Find eduwonkette's complete archives prior to Jan. 6, 2008 here.)

« Guest blogger Betsy Gotbaum on: The Future of Mayoral Control | Main | Vanity Fair »

No Child Left Behind: Looking Back, Looking Forward

soapy-maggie.gif
I'm knee deep in old NCLB documents, and ran across the Department of Education's NCLB song. NCLB represented not only a major shift in federal education policy, but an embrace of policy/PR boosterism that's enough to make all of us giggle (Remember Armstrong Williams?). Back from 2002, here are the NCLB lyrics:

We're here to thank our president,
For signing this great bill,
That's right! Yeah,
Research shows we know the way,
It's time we showed the will!
No matter how catchy the ditty, a song can't carry a fundamentally flawed law. That's where Tom Toch and Doug Harris come in. They've penned a thoughtful commentary in this week's Ed Week about the future of NCLB (Salvaging Accountability). It's an important one, because it recognizes that NCLB conflates the school's contribution to student learning with what students bring to the school to begin with. Essentially the argument is that:

1) "It’s critical in any accountability system that the metrics used to judge performance reflect accurately the contributions of those being judged."

2) "As a measure of school performance, however, [the NCLB] snapshot strategy is flawed. Because student populations vary greatly from school to school, and because family income, parental education, and a host of other non-school-related factors have a major influence on students’ learning, some schools have to improve student achievement a lot more than others to get their students up to state standards. The federal law is unforgiving of such schools. As a result, it gives an unfair advantage to schools with students from privileged backgrounds, and it fails to measure what matters most: how much students learn during the school year."

3) The Department of Education's Growth Model Pilot offers little improvement over the current rating system because it relies on a projection model - i.e. are students on target to be proficient in a 3 year window? - rather than a true growth model.

4) The new NCLB should dump the projection model, and focus its sanctions on schools that are both low in terms of their growth, and low in terms of their proficiency. And there's no reason to wait for reauthorization - this could all happen via regulations.

No commentary can do it all, so here are some issues to ponder for their next round. The goal of Toch and Harris' proposed system is to make measurement of school performance a more fair and effective enterprise. Why not take the leap and dump 100% proficiency altogether? That way, we could narrowly tailor our sanctions to schools that are low-performing compared to the schools we already have.

And if we're going to go full throttle on value-added models, we can't just punt the measurement problems. For example, Toch and Harris write, "value-added calculations have larger margins of error than NCLB’s proficiency ratings, but because they measure what’s most important in judging schools—student learning gains—their statistical shortcomings are more than worth tolerating."

A poorly designed growth model is no better than the poorly designed proficiency model that we have now, and no one knows this better than New Yorkers. Value-added systems that have literally no relationship between two years' value-added measures are still bad public policy. In short, beware the silver bullet.

TrackBack

TrackBack URL for this entry:
http://blogs.edweek.org/cgi-bin/mt-tb.cgi/5041.

Comments

I question the premise that sanctions should be imposed at schools where there haven't been sufficient improvements in test scores - no matter how they are measured.

Isn't this an even more severe form of high stakes testing -- but at the school level, likely to lead to the sort of test prep and cheating that you've written about in other contexts? What evidence is there that sanctions help schools improve actual learning? Can you give some examples?

Leonie is correct.

The danger is for "value added" to tend toward "valueless addition." That is of course an exaggeration - but surely no more so than a claim that a mostly multiple-choice standardized test, perhaps with some open-ended items attached, can enable an adequate assessment of what students should know and be able to do. Those are the very tests that are used for "growth or "value" added measures. This point seems to be lost most of the time by proponents of such measures, even cautious proponents.

If such tests are used with high stakes attached, then there is every reason to believe the same sort of narrowing curriculum to test prep that we see now under NCLB will continue. If anything, educators have been even more concerned over curriculum narrowing than the unfairness of measure only cohorts, not growth.

The Joint Organizational Statement on NCLB (now signed by 147 national organizations) calls for growth indicators but also for multiple forms of assessment. The Forum on Educational Accountability's Expert Panel on Assessment concludes assessment systems can do both, and that local and classroom measures can and should be incorporated into assessment systems. The two documents are available at www.fairtest.org and www.edaccountability.org.

Leonie and Monty are both correct. Combine this post with the Core Knowledge post on "re-routing," and new insights could emerge. They certainly started an interesting conversation at my school today. Some schools are engaging in the absurd process of re-routing where neighborhood schools get credit for scores of students who attend magnet schools but live in their neighborhood. But we used to do the equally absurd form of re-routing where our neighborhood school was held accountable for students referred to alternative schools for disciplinary reasons. But are either approach any more absurd than the normative approach of holding neighborhood schools accountable using the same rules as the law requires for magnet schools? Regardless of socio-economic factors, there is no comparison between a neighborhood school and a magnet school. So how can you use the same accountability system? (Presumable, a good accountability system follows the dictum, "you get what you measure." If magnet and neighborhood schools need different solutions, why try to devise the same accountability for the two?) Toch and Harris do a good job of explaining the drawbacks of their proposal. In fact, when you take into account their disclaimers you have to wonder why their ideas are packaged as a salvation of accountability. But many concerns would decline if we weren't trying to hold every school accountable using systems that are incapable of accuracy on that level and which invite trickery. How much more effective would Toch's and Harris' proposals be if they were holding entire systems accountable as opposed to schools? Certainly, my concern about the inherent impossibility of devising an accountability system that is fair and meaningful for both neighborhood and magnet schools would disappear. Given the underwhelming results of NCLB-type accountability, and the financial challenges we face, reasonable people should be able to agree that a system like Toch's and Harris', if it was applied to groups of elementary, middle and high schools could be valuable. A system along the lines of Toch and Harris would show whether our urban schools were adding value comparable to their suburban neighbors. It would reduce the incentives of individual administrators following all those counter-productive test prep policies. Perhaps it would deter entire districts from engaging in excessive test prep, for instance. Since we all would have the option of "hanging together," it might encourage educators to consider the best interests of their students. (if an entire district's elementary teaching force banded together and refused to comply with policies that damage kids, would they fire them all?) My district has followed the same the instructional "reforms" that consultants have pushed throughout the nation since NCLB. In secondary schools, they had produced significant increases in student performance in magnet schools but they have produced no gains in neighborhood schools, while increasing the drop out rate. Rather then explain my theories of why, I'll just repeat that neighborhood schools and magnet schools have very little in common, and they need different forms of solutions and different accountability systems. Again, to people who actually work in real schools, that's a no-brainer. These are just preliminary thoughts. But given the new financial situation, cutting back on test-driven accountability is a no-brainer. Let's see if we can turn this challenge into an opportunity.

How about trying a moratorium on high stakes testing and be surprised at what students actually learn? Bet they'd be a lot of excitement in schools across this nation.

This may be considered to be a risky idea, but I really do trust that students will learn (they learn all the time, like the rest of humanity) and that teachers will do their best to engage their students in discussing big ideas, open doors of opportunities, and offer students various ways of seeing and understanding.

Right now, far too many have been snookered into thinking that doing well on high stakes tests means that something important was learned. This perspective is limiting, like high stakes testing limits.

I would also like to suggest that money used on testing be redirected to fund schools in impoverished neighborhoods so that these schools on a par with schools in the more affluent neighborhoods. Imagine going to school every day in a school building (environment) that is falling apart and does not nurture the soul.

I agree with Leonie, Monty, and John. I too am concerned about the narrowing of the curriculum. Thus, my suggestion to have a moratorium on high stakes testing and teaching to the narrow, limiting standards. The only two standards that are important for the 21st century are the ability to: (1) think out of the box, and (2) question the status quo and authority. These two attributes are what leads us out to create new knowledge.

As it stands, high stakes testing and standards by committee cost millions of dollars, and have caused this nation to at the best to remain static and at worst to go backwards.

Let's take this window of opportunity to turn thing around and move beyond the manufactured crisis that got us here in the first place.

it fails to measure what matters most: how much students learn during the school year

But that's not what matters most. What matters most is how many kids learn to read and do basic maths.

Why not take the leap and dump 100% proficiency altogether? That way, we could narrowly tailor our sanctions to schools that are low-performing compared to the schools we already have.

So it's okay if kids fail to learn to read and do basic maths, as long as most of their fellow students at their school are learning?

100% proficiency I agree is impossible, human messiness always intervenes. But I don't see why you want to focus on schools rather than individual students.

Post a comment

Ground Rules for Posting
We encourage lively debate, but please, no profanity or personal attacks. By commenting, you are agreeing to abide by our user agreement.

The opinions expressed in eduwonkette are strictly those of the author and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Get RSS

Get eduwonkette delivered by e-mail. Enter your e-mail here:

Delivered by FeedBurner

Advertisement
Powered by
Movable Type 3.34

EW Archive