Rothstein Interview Part 3: Obama Faces Tough Questions
Last week I posted Part 1 and Part 2 of a four-part interview with author Richard Rothstein. Today I am posting Part 3, focusing on tough questions President Obama must face if he is to live up to his goals of improving educational outcomes.
3. You quote President Obama as being critical of the way NCLB has narrowed the curriculum to focus on tested subjects. Are there indications that steps are being taken to reverse this emphasis?
During the election campaign, President Obama said that NCLB “has become so reliant on a standardized test model that…subjects like history and social studies have gotten pushed aside. Arts and music time is no longer there. So the child is not having the well-rounded educational experience I benefited from and most in my generation benefited from.” We must change NCLB, he said, “so that the assessment is one that takes into account all the factors that go into a good education.”
To date, neither the president nor Secretary of Education Arne Duncan has indicated how the administration plans to apply this insight. It is more difficult than it looks. Any accountability system that uses incentives to improve only some of the goals of education will inevitably undermine the well-rounded education that President Obama supports. Rational educators will de-emphasize curricular areas for which they are not held accountable, to increase emphasis on those for which they are rewarded or punished. Thus far, most Washington discussion about "fixing" NCLB has stressed improving how we assess math and reading – for example, by using gain rather than level scores. However, even if math and reading assessment were improved, holding schools accountable only for math and reading and not for “all the factors that go into a good education,” will necessarily result in continuing to “push aside” arts and music, history and social studies, science, and other curricular areas.
4. There seems to be a consensus in Washington that NCLB can be fixed by making schools accountable for gains in test scores, rather than absolute targets. Will this be a meaningful change?
You are right; this consensus does seem to have captured Washington, but is ill-considered.
There is a conspicuous conflict between a desire to measure schools by their gains, and a continuing belief that all students can eventually reach the same proficiency point. NCLB requires that students with varied backgrounds and disadvantages must pass standardized tests by 2014. Although the Washington consensus seems to acknowledge that the date should be pushed back a little, no date makes sense if schools are to be evaluated by their gains. Advocates of using gain scores for NCLB accountability do not seem to have abandoned the idea that all students should reach the same level, but maintaining both standards simultaneously is ludicrous.
Should schools with more disadvantaged children, where present scores are lower, be expected to make faster gains than schools where present scores are higher? You might think so, because they have more opportunity to make progress. But perhaps students with more skills can apply them more effectively to learn even more. If so, then schools with fewer disadvantaged children, where present scores are higher, would make faster gains and the score gap will increase. Policymakers who advocate using gain scores to measure school effectiveness have not yet explained how they think such expectations can be adjusted.
Experts do worry about several technical impediments to using gain scores for NCLB accountability. One is that most states do not yet have data systems that can link a student’s test scores in successive years; another is that gain scores must be based on two successive annual tests, compounding the unreliability of each; another is that school evaluations based on gain scores must ignore large numbers of students who switch schools at the beginning of or during a school year; yet another is that schools, even after controlling for demographic factors, may not enroll representative collections of these demographic groups.
But these technical discussions avoid the more serious issue I discussed above - an accountability system based on easily measured subjects will inevitably result in narrowing the curriculum, because educators held accountable for math and reading will rationally redirect their effort away from other areas. Holding schools accountable for math and reading gains instead of math and reading levels will do nothing to solve this problem.
Nor can we solve it by testing in other subjects. Some don't lend themselves to standardized testing. Whether schools are teaching students to work cooperatively, to exhibit good habits of civic participation, to resolve conflicts nonviolently, to appreciate the arts and music, or to develop healthy exercise and other lifestyle habits, cannot satisfactorily be assessed by either gain or target scores on paper-and-pencil tests alone.
Reform of NCLB’s accountability design requires more than improving our technical capacity to evaluate the teaching of math and reading. It requires development of systems, requiring qualitative judgment along with testing, that give schools incentives to deliver a balanced curriculum.
5. Can you explain Campbell’s Law? Why do you think the crafters of NCLB ignored this principle when designing their program?
The great methodologist, Donald T. Campbell, studied President Richard Nixon’s “war on crime” in the 1970s and concluded that “the more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.” Campbell found that when police departments were held accountable for reducing crime rates, reductions were achieved by manipulating statistics, not by better policing.
Even before Campbell’s study, and certainly afterwards, social scientists have observed that institutions other than schools are invariably corrupted by accountability only for narrow, quantitative performance measures. Grading Education illustrates this with examples from many public and private policy fields. For example, workforce training agencies, held accountable for placing unemployed workers in jobs, reduced educational programs leading to high quality, long-term careers, instead emphasizing placement of large numbers of workers in short-term, unskilled jobs that boosted agency success statistics. Cardiac surgeons, held accountable for patient survival rates, demonstrated superior performance by refusing to operate on sicker patients. U.S News and World Report evaluates the quality of colleges by the percentage of applicants who are accepted; colleges have reduced their percentages (and boosted ratings) by waiving application fees for unqualified high school graduates. The narrowing of curriculum under NCLB, the “teaching to the test,” the opportunistic focus of instruction on students who could boost “adequate yearly progress” statistics rather than on students who most need attention, all have been foreshadowed by similar experiences in other fields.
I really don’t know why the Bush Administration, Congress, and policy advocates who crafted NCLB ignored this overwhelming body of experience, especially because, even in the private sector, institutional accountability rarely relies on simple quantitative indicators. Instead, qualitative evaluation in the private sector is commonplace.
What do you think? Do you see the narrowing of curriculum Richard Rothstein describes? Has our emphasis on test scores distorted our educational system? What do you think of the steps President Obama has taken thus far?