« Time to Enlarge the Public's Imagination | Main | Reforming and/or Busting Unions? »

What Do We Know About Improving Schools?


Dear Deborah,

You are undoubtedly right that what we have been calling "reform" is not producing better educated young people, but I don't think we can move on and forget about what is happening now. We have to keep talking about why current schemes don't work because so many politicians and journalists are convinced that they will work if we just keep plugging for another five or 10 years. If a journalist finds one school that seems to be doing incredibly well, that is considered an existence proof, demonstrating that every school can do incredibly well. The fact that almost 30,000 schools (out of 90,000) did not make "adequate yearly progress" under the No Child Left Behind act doesn't register with the public consciousness. The fact that the number of "failing" schools increased by 28 percent in the last year alone is not newsworthy, except in Education Week. What happens by the year 2014, when nearly 100 percent of all elementary schools in California will be deemed "failing" schools ?

The public will be riveted by a story of a little girl who falls down a well, but will shrug with indifference when they hear or read reports of the massacre of hundreds of thousands of people in Rwanda. I guess that is why an uplifting story of one turn-around school gets so much more attention than little-noted, humdrum reports about thousands of public schools that are on track to be restructured or closed.

Speaking of turn-around schools, I want to draw your attention to a research publication that I received last week from the National Center for Evaluation Assistance and Regional Assistance, which is part of the Institute of Education Sciences, which is part of the U.S. Department of Education. Whew! The publication is titled "Turning Around Chronically Low-Performing Schools," and I opened it with great interest. My scan of research to this point has convinced me—at least for now—that no one has a formula for turning around chronically low-performing schools. I have been amused to see that several universities have opened programs to train "turn-around specialists," as though there were a set of lessons and techniques that anyone could learn to do.

So when I read this publication, I discovered that it contained four recommendations:
1. Signal the need for dramatic change with strong leadership.
2. Maintain a consistent focus on improving instruction.
3. Provide visible improvements early in the turn-around process.
4. Build a committed staff.

What astonished me was that each of these recommendations was accompanied by a notice that the level of evidence supporting it was "low." I wondered why in the world the U.S. Department of Education (and its related entities) was publishing a set of recommendations where the evidence was so slim that they were not willing to vouch for their effectiveness.

What always troubles me about recommendations of this kind, even when they do have evidence to support them, is that as phrased they are bromides. Every one, I assume, wants strong leadership; everyone wants better instruction; everyone wants quick improvements; and everyone wants a committed staff. Between having these goals and having a way to get there is a huge gulf.

This, it seems to me, is a problem with education research. If all they can offer practitioners are maxims and bromides, no wonder research gets a bad name. It is even worse when the bromides do not actually have research evidence supporting them. And worst of all, when the U.S. Department of Education has the chutzpah to publish findings for which there is "low" evidence.



What makes it really painful is when you're in a city like Providence which is stuck in the cycle of restarting that process every two to three years.

On the other hand, there is pretty solid evidence that continuing as we have been doing results in, well, what we have: a national system that has allowed other countries to race ahead of us--claiming the spot we once held of the cultivating best and the brightest by cultivating all. Other countries have accepted the challenge to educate all--and some have been doing an exceptional job of it.

I don't have a big problem with presenting weak evidence if it is in fact the evidene that we have. And I recall that the early criticism lobbed against the Dept of Ed was that What Works wasn't finding much in the way of sound evidence that ANYTHING works. At this point, I suspect that both conditions say more about the state of educational research in this country than anything else. But educators have been clamoring that the Dept of Ed stop telling us what doesn't work and give us SOMETHING we can do.

There is some movie--I believe it stars Rosalind Russell as Sister somebody or another, who did some pioneering work in polio. She started out as an outpost nurse in Australia badgering a doctor to tell her what to do. At that point very little was known about polio, it was frequently fatal and those who survived were generally crippled for life. There is a key scene in which the doc is telling her to just do what makes sense (after all--what could she hurt). Well, as it happens, what made sense to her confounded the generally accepted practice and she spent a good bit of her career trying to convince the (male) medical establishment to take a look at her evidence. As it turns out, her left turn--when the rest of the medical world was turning right--was a breakthru, once she could convince someone to pay attention.

The organization of education may be at a similar point. There is the conventional wisdom, which isn't working very well, and a few pioneers who have gone another way--and may have stumbled on to something. But I don't think it serves us well to simultaneously reject any organizational knowledge garnered from business, refuse to apply any weakly supported changes in education because they are weakly supported, and steadfastly cling to what we are doing (or wish we were still doing).

There are processes that involve determining a hypothesis about what will bring about improvement, testing the hypothesis and evaluating the results. You can call it science or you can call it Deming. Even with the best of solid research about "what works," there is always going to be a certain amount of improvement tweaking at the local level--provided improvement is what one is aiming for.

The first lyrics that Barack Obama heard during Sunday’s concert at the Lincoln Memorial, “We Are One,” were Bruce Springsteen’s “Can’t see nothing in front of me ...” In his brilliant account of the 300 firefighters running up the World Trade Center, they were pushing a 60 pound stone up those eighty-something flights of stairs. Their courage came from the ½ mile line of hose, or their loyalty to their comrades, their commitment to their profession’s calling, and visions ahead of family and community and faith.

No mention of made of a blueprint for success and a guarantee of victory.

The civil rights leaders of the sixties like Joseph Lowry said they had not envisioned a Black president in their lifetime. What kept them going without a formula for 12 year plan for 100% success?

Paul Tillich challenged us to take a “leap of faith into the unknown.”

According to some accountability hawks, all of the above would be unfit to join the civil rights movement of the 21st century. Michelle Rhee and others say they don’t want educators who don’t believe that we can obtain measurable transformations in learning, even without resources. If you don’t believe a) they we already have proven blueprints for success, and b) they require us to measure all learning and achieve it by holding everyone accountable to hard statistical models and schedules, then you have “low expectations.” Diane’s post would bar her from the battle. Deb’s opinions mean that she is “status quo.” Anyone who doubts the one true way of the “reformers” - who they refer to as the reformers, without the quotes - should be barred from the Obama administration.

David Berliner rightly says that these “reformers” are demanding “a civil rights movement on the cheap.” Tillich would add that demanding guarantees of victory would be “cheap grace.”

Obama seems to be following FDR who frankly said that we should experiment and admit it when our experiments fail and then try something else. But of course, that is why Obama scares Rhee.

I understand how the politics of the 80s and 90s gave rise to the accountability movement, giving liberals a chance to sound as tough as the Republicans. NCLB-type accountability has largely failed. Why not lift our sights to challenges that are worthy of a democracy and education? Why continue to claim you are on the side of poor children and then ban the values of the liberal arts and free expression and take the risk of condemning kids to the disrespect of excessive test prep, given all of the evidence that children and teachers can not succeed without a respectful learning environment?

Why can’t the accountability hawks even acknowledge the existence of trade-offs? Nobody is perfect, but they won’t even allow discussion of the possibility that their solutions produce harm as well as good.

I don’t believe I ever have a day at school without making a decision that causes harm as well as good.

Still can’t answer the question of why can’t “reformers” tolerate debate? Why not challenge Americans to join a battle without a guarantee or blueprint for success. Why not challenge educators to think anew and create new experiments? As we tackle new frontiers, why not admit that we don’t know how it will turn out?

The truth is, we are largely flying blind in our effort to fulfill the promise to kids who attend our poorest neighborhood schools. Obviously, there are lessons from KIPP and TFA and selective schools. But to say that those outliers give us a formula for success is silly.

NCLB-type accountability supporters often condemn the practical wisdom of teachers saying that we’re just making excuses. Had they been in the battle long enough, they would understand that we’ve always had a surplus of excuses and/or explanations and we didn’t need a young generation of “reformers” to prompt us to make more. Primarily, we’re not frustrated by people who believe that our opinions are excuses. We are frustrated that their simplistic blueprints often make things worst.

We didn’t sign up to sacrifice our lives and we should not be compared to heroes who have. But, we did sign up for a career where much of our efforts still result in the stone rolling back down the mountain. It’s our job to suck it up, and then start pushing again. And we’ve also known that plenty of Rightwingers would be taunting us for not achieving utopia. The new frustration is that now many of our comrades on the Left have joined in McCarthyism.

This is already too long, but it seems to me that the most vocal of the civil rights leaders who demand the silver bullet of accountability are veterans of the 70s struggle, that took place disproportionately in the courtroom. They could use numbers alone as evidence of discrimination. In issues ranging from affirmative action to school equity to comparable worth. At times their logic approached social engineering, but at the time I would not have said that. I respect their efforts, even though their success was incomplete. Why can’t they show the same respect for educators? They did not have foolproof solutions then and we in schools don’t have answers now. How come they can’t admit that if they had spent as much time in the classroom as we, perhaps they might recognize that we also have good intentions, evidence and logic on our side? Why are so many newcomers convinced that they alone hold a monopoly on truth?

So Diane, your post was great. We don’t know nearly enough. And that will not bother me one little bit, when tomorrow I head back to school.


Loved your use of "bromide."

If we consider the source - the US Department of Education - and who has been in charge of this bureaucracy for the past eight years, it says much about these obvious recommendations.

The more I read and think about reforming our schools, the more I am convinced it comes down to the teacher and the twenty or so kids in each class. It's Interpersonal Relations 101. I’m also quite certain there is no one magic pill that's going to make all schools functional. What might work in class X might be a total failure in class Y. Will we ever be able to come to this conclusion via educational research? That would be some study.

Is the teacher: compelling/passionate/interesting/entertaining/intelligent/compassionate/caring/rational/inspiring/pragmatic etc., etc., enough to move (most of) the kids in that class to want to learn? If the answer is yes, good results will follow. If the answer is no, then you can have a mild to full blown disaster on your hands.

Rosalind Russell as Sister Kenny


I am not surprised that IES concludes that the "turn-around" evidence is weak. The current state of education research is insufficient for us to make definitive statements about the best way to teach math or science--leading to "math wars." (The evidence is stronger in reading but the controversy seems to remain.)

How much more difficult a task to "turn-around" a whole district! And, like Sister Kenny, when confronted with a problem for which there are not proven research based therapies we still have to decide to do something that "makes sense." Hopefully, however, well designed research around these interventions will continue because sometimes what "makes sense" does not help, and sometimes it hurts.

In the early 1940's standard medical treatment for premature births was oxygen enrichment therapy. The therapy was known to reduce mortality and brain damage. The first case of blindness in these premies was reported in 1941. Over the next ten years more than 10,000 cases of blindness were noted. There were about 50 competing hypotheses to explain why these cases were occurring. 25 studies were conducted, none of them were randomized controlled trials (RCT's), but were not able to pinpoint the cause.

Some anecdotes, first from Australia and then from France and the USA, seemed to point to oxygen therapy as the cause. An RCT was conducted at a hospital in Washington, DC. Seventy six premies were either randomized to the standard oxygen therapy treatment or to a "low-flow" oxygen treatment. The design of the trial was subverted by the night nurses who turned up the oxygen flow for the "low-flow" premies after the doctors went home for the night. They did this because they "knew" that oxygen could not be the cause of the blindness because...well duuh...its OXYGEN! Even so, the low oxygen group showed a lower incidence of blindness and a larger 800 baby trial was conducted. Incidence of blindness in the high-oxygen group was 23%, in the low-oxygen group 7%.

Good research is needed because sometimes it is really, really hard to know what we know. There are many examples of advances that seem "obvious". But it is important to realize that before good research made them "obvious" it was often not clear whether the intervention was good, bad, or indifferent.

Here's hoping that IES keeps up the pressure for rigorous education research under this new administration. It is not that everything they fund should be an RCT--no way. But it ultimately takes this kind of research for us to understand what works, what doesn't, and what just might be toxic!

Diane -

I think the key thing about these four "bromides" is not what they say, which as you indicate is pretty universally agreed, but what they do not say, what we are not doing to improve a school because we are trying to focus on these things. So...

1. Signaling the need for dramatic change with strong leadership, may be precluding getting students and their parents more involved in the turnaround process. Leadership can be directive/authoritarian or facilitative/partnering. This statement may be emphasizing the former at the expense of the latter.

2. Maintaining a consistent focus on improving instruction, may be preventing a move toward giving students the opportunity to learn more high-level skills in areas of their unique interest, rather than being transmitted a one-size-fits-all authorized curriculum.

3. Providing visible improvements early in the turn-around process may be precluding doing the extra work of engendering a more democratic change process that would involve students and their parents. It solidifies school administrators as the true and only school decision-makers and students, parents and teachers in the role of spectators.

Cooper Zale


Every political stripe except libertarian invokes FDR to back up their "experiment"- a way of saying politically controlled. Reagan loved FDR. Although you may find some split among neocons, bigwig publisher Conrad Black defends FDR.

Not one acronymic morsel in the veritable alphabet soup of FDR's New Deal etc. succeeded in moving America out of the Depression.

The bloody war that FDR supported, against the overwhelming sentiment of the American people I might add, only hurt the economy and brought massive death, destruction and- what is oft overlooked by WWII hagiographers- victory to the most prolific killers of the century: the Bolsheviks.

It was not until FDR's experiments in war socialism, and what might be called an American variant of Mussolinism, came to an end after the the war that massive economic recovery ensued.

If Obama follows in FDR's footsteps (the way that Bush followed in Hoover's?) then Americans will be in for a very very long period of unnecessary suffering indeed.

In that case, let me be the first to throw a shoe, albeit a cyber one, at Obama and his 'experiments'.

I spent a little time looking over the publication mentioned, and indeed it gives one pause. It recommends spouting a lot of rhetoric and firing a lot of people. It also recommends, perhaps less directly, that we have good intentions. It does not explicitly recommend the use of magic or crystal balls, but I think that is implied. It doesn't tell us how else we are going to make visible improvements early in the process.

The person Margo refers to is Sister Kenny. I have long been aware that in the medical world she is credited with making important contributions to the care of polio victims, and that she had to buck conventional medical wisdom to do so. All I know beyond that comes from Wikipedia. "Sister", it turns out, was a military and/or occupational title. I always assumed she was a nun, but apparently not. Are there counterparts to Sister Kenny in the educational world? And can we benefit by their insight? Undoubtedly there are, and I think we benefit from them everyday. But I don't see any great changes coming from that direction.

I think a big problem in education is that conventional wisdom is very ill defined and fragmented. The pronouncements of ed school can be called conventional wisdom. A rejection of the pronouncements of ed school, it seems to me, has an equal claim to being considered conventional wisdom. In that perspective I'm very much a rejectionist of ed school thinking, and I don't think I'm in a minority.

I think every teacher who actively and systematically uses his/her common sense, intelligence, communication abilities, and social skills is like Sister Kenny, at least in the sense of maximizing human potential. These people (and I like to include myself here) are the work horses of the educational world. Their many conventional wisdoms may be fragmented and ill defined, but they work.

By my perspective then, education improvement is going to be hard to come by. The recommendations for spouting a lot of rhetoric and firing a lot of people seem counterproductive.


I took a look at "Turning Around Chronically Low-Performing Schools" and was not cheered by the checklist on page 9. The last category in particular set my sirens off:

Recommendation 4: Build a committed staff:

__ Assess the strengths and weaknesses of the staff. Identify staff who are not fully committed to the school turnaround goals or who do not have the qualifications to carry them out.

__ Redeploy staff members who have valuable skills but are not effective in their current role.

__ Replace staff members who actively resist the school’s turnaround efforts.

__ Recruit new staff who have the needed specialized skills and competencies for positions in the school—such as interventionists, reading specialists, and mentors and instructional coaches.

What if the change in question is unsound or only partly sound? What if it is being implemented district-wide, without consideration of the drawbacks or problems? What happens to those teachers who are replaced or reassigned? What happens to the school itself, when one fad after another passes through?

Yes, it is important for the staff to support the school's mission and principles. But it is a mistake to confuse those with the school's game plan for change. Change is by nature transitory; it is often superficial and shrill. Schools need something humbler, lovelier, and stronger than any given change, something that will outlast the noise.

Diana Senechal

and a little tweak there...

Recommendation 4: Build a committed Bolshevik Party:

__ Assess the strengths and weaknesses of the staff. Identify staff who are not fully committed to the Party's turnaround goals or who do not have the qualifications to carry them out.

__ Redeploy staff members who have valuable skills but are not effective in their current role.

__ Replace staff members who actively resist the Party’s turnaround efforts.

__ Recruit new staff who have the needed specialized skills and competencies for positions in the Party—such as right-thinking interventionists, re-education specialists, and ideology mentors and instructional police.

Hmmm. Did Stalin really say "Where there are people there are problems. No people, no problems."?

Notably absent from this synthesis,"Turning Around Chronically Low-Performing Schools," is any review/evaluation of the selection criteria for principals/educational leaders. In higher ed., the curriculum for educational administration often emphasizes policy, and not research, possibly having a significant bearing on the caliber of our leaders.

I also noticed recommendations for principals to spend more time in classrooms. I'm guessing if principals were interested in doing this, many would not have left the classroom in the first place. Interesting...

@Paul: "there is no one magic pill"

You are right on. What works in Brooklyn will not necessarily work in Idaho.

Remember "Midnight Basketball?" Some adults in an inner city opened a gym and hosted basketball games for young people as an alternative to hanging out on the street. And it worked.

So federal money was made available to start Midnight Basketball programs around the country.

But it turned out adults investing in young people was more important than a "program."

In my comment yesterday I made fun of the publication Diane wrote about. I think we should make fun of the "spout a lot of rhetoric" part. But, thanks to comments by Diana, reason, and others, I got to thinking about the "fire a lot of people part". That's not funny. Every child can learn, but not every teacher! Is that going to be our slogan? If a political party put out an idea like that in a political context it would be subject to scathing criticism from the opposition, as indeed I think it should be. "Fire a lot of people" is a draconian proposal. It ought to be treated as such.

Of course the idea of firing people when needed has to be considered. Indeed we come across that idea rather frequently in the blogs. In any field of endeavor there are times when it has to be done. I accept the idea that some teachers and some administrators really ought to be fired for the greater good. But it should never be taken lightly. With few exceptions it is a very serious trauma to the person who gets fired. Firing may be justified at times when there is a clear and proven alternative, when it really is a “solution”, not just an experiment. But remember that the alternative we are talking about here is rhetoric and good intentions. That is not a solution to anything.

Jay Mathews new book "Work Hard. Be Nice: How Two Inspired Teachers Created the Most Promising Schools in America " is scheduled to be released today. KIPP's primary hurdle to large scale adoption is they have no magic bullets,they spend more focused time teaching. If you reallly want KIPP's excellent results no more 180/6hr day school years. The KIPP school year looks as if they read "The learning Gap" by Stigler and borrowed the day/week/year schedule (240 days per year)employed in Japan. I'm waiting for a graduate student in statistics to tell us how many KIPPs there could be and how many more Ghetto kid's would be succeeding if we wanted to actually close more of the achievment gap.

John S.,

If Levin and Feinberg had read The Learning Gap they would also have gone to great lengths to incorporate "lesson studies" into their schools as the schools in Japan did. To my knowledge KIPP schools do not employ this practice on any scale.

I don't think that we need a recipe for turning around schools, or helping chronically low-performing schools. As long as we are looking for such a recipe -- and many are -- the evidence will remain weak.

And we as educators know this. We can't take what worked over there and copy here. That's won't get us the same results.

But we we move out a level, we CAN bring what they did over there to here. It's not what they did that we need copy, rather it's how they figured out what to do. If we copy their approach we might end up with a very different answer or a very similar answer. Either way, it's the approach to deciding what to do that needs our attention.

And I think that there actually are some common steps that work -- though each needs to be defined/refined/localized.

1) Examine what is actually going on. (NCLB does take a big step in this direction with its disagregation of data by group and by grade, but there are a million areas that NCLB doesn't even begin to look at.)

2) Think about what we would like to happen. (NCLB skips this entirely, as do most reform paradigms, in my view. Getting everyone on board about the actual goals, understood in ways that influence how all the adults do their work, is key.)

3) Figure out a plan for how to get from where you are closer to where you want to be. (Obviously, NCLB ignores this.)

There are no answers in these three steps, rather questions that need to be answered.

Of course, that's also a different model of education as well. Some of us feel that schooling should give kids answers, and others feel that shcooling should help kids learn to ask and answer questions themselves.

I'd like to return to Diane's observation regarding the low evidence for each of the recommendations in the report. As she points out, the report itself acknowledges that the evidence is low. I suspected that educational organizations would overlook this and refer to them as "research-proven" or something similar.

Sure enough. If you run a Google search on "Turning Around Chronically Low-Performing Schools," you will see that a number of sites refer to the recommendations as "evidence-based." What does that mean? You can find all sorts of "evidence" for things that are not on the whole useful or true. If I begin a lesson with a shrill song, the children may stop talking and turn their eyes toward me. It may only work once. It may be highly annoying. It may even be bad for the ears. But I have my "evidence," correct? The approach is therefore "evidence-based."

Thus an idea of mixed merit quickly turns into propaganda. And if teachers are not allowed to question such things, the propaganda swells into that very shrill song--annoying, ear-piercing, and insistent on itself.

In a brilliant little book entitled The Unique Funcion of Democracy in American Education (1937), Charles Beard wrote:

"To be sure, education cannot be entirely divorced from immediate ends and objectives. Yet there is a center of gravity in education which is not the center of gravity in propaganda. The spirit of education differs from that of propaganda. In some respects, as in other matters, it is a question of emphasis, but the emphasis is fundamental. The propagandist deliberately refuses to present to present with all the fairness that human fallibility will permit the other positions or points of view which enter into competition with his own. He places the interest of his group above all other interests. His temper is dogmatic, not inquiring or reasoning. He puts forward opinions as established facts and closes his mind to new truths incompatible with his ends. If education could perchance endorse any of his designs, it could not proceed in his spirit or follow his methods without violating its trust. By its inescapable obligations, it has other functions to discharge."

Indeed it does.

Diana Senechal

Oops, sorry, the title is The Unique Function of Education in American Democracy. Sorry about that!


I think that you are absolutely right to focus on the problem solving process. However, I would argue that NCLB does not overlook this planning component--in fact it mandates it. Like many aspects of the NCLB, however, the specifics are left up to the states and locals to work out and enforce. And like IEPs required for special education, having a plan on paper doesn't necessarily translate into implementing the plan in real life. AND, it's pretty easy to fill in all of the required squares, lines and boxes with stuff that doesn't add up to a plan that moves from where we are now to where we want to get there, how we intend to move and how we will know when we get there.

This is my biggest frustration with the folks who focus on the sanctions that before schools/districts after they have failed for 5-7 years to make the minimum required levels of improvement. They should at least have amassed an excellent record of what they tried and when and whether and to what degree it made any difference. Most don't--but during that time a kindergarten student has moved on to middle school in a school only marginally better than when they started out.

Bureaucracies--of which school systems are a big one--run best when things are quantifiable and predictable. This is one reason they like quantitative evidence even if they often use it dogmatically rather than analytically. But as Diana S. says, education has "other functions to discharge" which are not always evident from such "evidence" no matter how inconvenient it may be to bureaucratic functioning. One of them is recognizing the dogma which often underpins bureaucratic convenience.

Well said, Tony, though I don't want to take credit for Beard's words!

I learned about Beard's book (The Unique Function of Education in American Democracy) from Diane's The Troubled Crusade. Her description made me want to locate and read the book, and so I did.

The official author is the Educational Policies Commission. Beard wrote the original manuscript. The commission made some revisions, but it is clearly the work of a wise, learned, inspired individual, not a group.

Comments are now closed for this post.


Most Viewed on Education Week



Recent Comments