« Expanded Learning Time | Main | NCLB: Thoughts for Congress »

Closing the Measurement Gap


In a recent Education Week Commentary, sociologist Jennifer Booher-Jennings of Columbia University writes that the federal government endorses entirely different standards for measuring performance in medicine than it does for measuring public schools.

In rating the quality of hospitals, federal agencies make "risk adjustments" for hospitals that treat sicker patients—the idea being that that those institutions shouldn't be penalized when more of their patients die because of factors beyond their control. But in measuring the quality of schools, no such risk adjustments are made for factors such as poverty or student mobility, Ms. Booher-Jennings contends.

What do you think? Should schools' performance be adjusted for risk factors? If so, which risk factors would you include?


It's at least as good an idea as what we have in place now with NCLB. It might help pull it back to a more reasonable middle point if implemented. But don't hold your breath. Ms. Booher-Jennings writes that " Sound public policy should be able to distinguish these two conditions"(between a child failing and a school failing). The problem is that it is NOT "sound public policy"; it is politics. Reason has little to do with it.

Many risk factors are already "factored" in. The difficulty is in implementing the use of these risk factors. Exceptions exist for special education, English language learners, and "achievement gaps". For education, the challenge is to overcome and eliminate these risk factors, which is not a realistic expectation. A school, for instance, that serves a high special education population is likely to continue serving that same type of population regularly withmuch smaller achievemnet gains than another school. An area that has a high immigration rate, or migrant family population will regularly be adding new English language learners as cohorts of students move through the program. Current policies do not account for students moving into risk groups as others pass out of the same groups.
In medicine, federal agencies trust the doctors and hospital administrators to be giving them accurate information. In education the federal agencies question the motives of the teachers and administrators when they raise the issue of risk assessment.

Having done a stint in health care administration, I can testify that federal agencies place no greater trust in the information coming from hospitals and doctors than from teachers and schools. And certainly, in medicine the problem is compounded by the fact that there are so many different payers making demands.

I think that beyond the question that Ms. Booher-Jennings begs, that of whether and by what factors the success of schools ought to be risk-factored, is the question of whether we want to live in that kind of world, or country. Given that the available schooling is still largely a function of where parents can afford to live, are we willing to accept schools that produce less-well-educated students, on the whole, than others? The issue of whether the student failed or the school failed, particularly at the elementary level, really strikes me as adult abdication of responsibility.

Perhaps, as an alternative, we could develop school assignment plans that would equalize the risks (since the risks seem to be resident within the students, this could be done through an evaluation at kindergarten entry). Or maybe we could just use the available rating systems to place resources where the risks are.

Placing resources where the risks are is a phenomenal idea, and it would be useful in narrowing the achievement gap that NCLB was supposed to fix. However, the resources typically go to where the parents can and do provide needed resources instead of where they are not in any other way available.. One side of our district complains about not having an asphalt track while the other side needs classrooms, teachers and books.

There is a critical need for this or any other system
that recognizes the socio-economic differences that exist across schools within a district & across districts.Holding all schools to the "same high standard" is unrealistic if not stupid. School should report in a value added format even if the states continue a "status" reporting system.It would be interesting for educators to read CLASS & SCHOOLS by Richard Rothstein.

Teaching students who are at-risk of failing due to appropriately identified disabilities (let alone the many students who are not yet identified or are misidentified), requires highly skilled and highly educated teachers. The current politically correct response-to-intervention model (RTI) supposedly eliminates the need to "label" students and focuses instead on evidence-based interventions in increasing levels of support. Regardless of whether this model is effective or not (and I believe it at best delays services to those who most need them and at worst denies students of their civil rights), the existing educational research base demonstrates that 1-5% of our students will not respond to any type of known intervention. Therefore, I feel we absolutely need a different kind of model that considers population risk factors. More than that, though, we need leaders who will stand up and support students with exceptionalities, their families, and their teachers. In my opinion, the biggest risk factor to student success is oppressive leadership. Not until we change the selection process for our school leadership will we begin to create true educational equality for all.

You are right on target! Good job!!!!!

I was shocked by the logic used in this commentary, and by the approach our hospitals take to evaluating their success. I agree with Ms. Booher-Jennings that there are some serious problems in our educational accountability system, and the way its findings are abused. But that doesn't mean that risk-adjusted measures should be the wholesale replacement for what we have now.

I see two big problems in Ms. Booher-Jenning's argument. The first can be illustrated by a thought experiment.

Ethnicity A has a heart disease mortality rate of 600 out of every 1,000 patients nationwide. Ethnicity B has a heart disease mortality rate of 200 out of every 1,000 patients admitted nationwide. In Smithtown, where 90% of the residents are Ethnicity A, 500 out of 1,000 heart disease patients can die and Smithtown Hospital will be rated a success story. In Johnsonville, where 90% of the residents are Ethnicity B, if 300 out of 1,000 heart disease patients die, their hospital will be called a failure.

Yes, I can see why some would argue for such a "risk adjustment" but that's the same thing as saying "You're more likely to die of heart disease" just because you were born in a certain ethnicity or social class, or grew up speaking two languages (regardless of diet, exercise, etc). Granted, I don't know which demographics hospitals are using in their adjustments, but the implications are frightening -- especially when transferred to schools.

Statistical norms aside, there is no inherent reason that being born into Ethnicity A should doom a child to scoring 20% lower than Ethnicity B, or that being born poor should keep children from growing up to earn Ph.D.s or six-figure salaries.

My second problem with Ms. Booher-Jennings' argument is that the missions of education and medicine are different. These hospitals are measuring mortality rates. Is the mission of hospitals to prevent all deaths? If so, 100% have failed miserably. On the other hand, NCLB (with all its problems) is exposing a conflict over what the mission of education truly is. Are we suppose to help only some kids become well-educated, or all of them? If we really mean all of them, what needs to change about the way we go about our work? Should K-12 education take more or less years depending on a student's aptitude? Why do we expect them all to learn at the same pace? The goals of medicine and education are quite different. There's no reason that they should be measured the same way.

When my older son was 14, he had braces on his teeth. He was supposed to stay away from sticky candy, soda pop and had to floss carefully. He had to be extra careful because he is diabetic (This disease affects dental health). He was not very cooperative and I didn't have the patience to keep after him. One day the orthodontist called me in to view a film on dental hygiene. I was so angry about having to sit through the film that I ordered the dentist to "just take them off now." I had to sign a statement attesting to the fact that the treatment was incomplete.

When the braces came off, I was horrified to see that my son's teeth were damaged; there were grooves that he has to this day (he's 36). I was so angry with the doctor for failing to alert me to the fact that Mike's teeth were being damaged. I chose another orthodontist for my second son. This son was healthier and more cooperative and therefore had a much better result. My husband and I, fearing another bad outcome, were much more cooperative also.

Of course now I know that my older son's bad outcome was due mostly to his lack of compliance and my husband's and my lack of care. Yes, the dentist should have informed me that our child's teeth were being damaged, but would you want to lay all the blame for my son's teeth on the doctor? In fact, I found out later that he had an excellent reputation with many success stories. Would we want to do "risk adjustments" before evaluating this orthodontist? I would hope so.

The fact is that almost all services for children (education, medicine, dentistry, counseling etc.)depend on the parents to partner with the professional who is providing the service. Many people would disagree with me, but I believe parents play the larger role in the outcome of any service. (e.g.doctor spends fifteen minutes prescibing a medication and the parent spends time each day for two weeks administering it and watching for effects.) In education, the research tells us repeatedly that the level of a child's achievement is directly related to home factors. In fact the only school factor that really counts is the quality of the classroom teacher, but that factor is secondary to home factors. This is no longer just opinion. We've had over forty years of research supporting this.

The only good thing about NCLB is that it has focused the spotlight on the glaring discrepencies in the education (formal and informal) of the privileged and the underprivileged. Finally we are hearing more about such factors as health,housing, preschool, parent education and language competency.

I believe that "everyone" knows the home is critical to the education of the child. Those who deny it are probably doing so for political purposes or private agendas. Or, like me with the dentist, they might want to blame the professionals for their own failures.

Should the evaluation of a school use risk factors? Absolutely. Which ones? I'd say health, quality of language by age five, education of parents, number of books in home, amount of time child is read to each day, number of hours of TV viewing each day,etc. For a great insight into reasons for the achievement gap, read The Meaning Makers by Gordon Wells. Another really good book is the one by Richard Rothstein cited above.

Yes, the child with untreated ear infections can learn, but will he learn as much as the kid who gets treated? Probably not.

In Vernon Jordan's autobiography, he wrote that his mother wanted him to succeed in school so she observed the child-rearing practices of the rich lady for whom she worked. If the lady gave her son piano lessons, Vernon got them too. If the rich child listened to his mother reading, Vernon's mom did the same.

If we really want to close the gap, we'll study the factors that influence high achievers and then try to provide some of those experiences for all children. Flashcards and phonics will not do it.

I think that before anyone jumps to the conclusion that evaluating schools on a sliding scale is acceptale, a great deal of research must be done. The latest research statistic that I have heard out of Harvard is that 70% of a student's success in school is based on the effectiveness of teachers and the experiences they create. It is vital that we do not commit a disservice by assessing and funding schools based only on their service records related to factors like poverty etc.

I think it's really important to understand the research. Of course the classroom teacher makes a huge difference. The education community has known for years that the most important SCHOOL factor affecting achievement is the effectiveness of the classroom teacher. However, the overall academic achievement of a student correlates more with socioeconomic factors than any other factors, including school. We have had this information for over forty years and it doesn't help to ignore it. It's expensive, but not impossible, to address these factors. Only then will we see a narrowing of the achievement gap.

Comments are now closed for this post.


Recent Comments

  • Linda/Retired Teacher: I think it's really important to understand the research. Of read more
  • Eric Conrad/ CT Teacher: I think that before anyone jumps to the conclusion that read more
  • Linda/Retired Teacher: When my older son was 14, he had braces on read more
  • Aric, Education Vendor: I was shocked by the logic used in this commentary, read more
  • Marilyn Bardill, Teacher-Librarian: You are right on target! Good job!!!!! read more




Technorati search

» Blogs that link here