« The Bible Makes A Comeback | Main | Religious Expression in Texas »

Making Policy Work

| 13 Comments

Education policy would greatly aid in the improvement of U.S. students' scholastic achievements if it borrowed a page from medicine and used more research-based evidence to evaluate programs and best practices, writes Jon Baron in this Education Week Commentary. Baron cites examples of programs that have been identified by well-designed randomized-control trials as being highly effective.

According to Baron, "The very existence of a few research-proven educational programs ... suggests that a concerted government effort to apply the rigorous methods used in medicine to education policy could fundamentally increase the effectiveness of such policy."

What do you think? Should education policy be more research-based? Could medicine provide a model?

13 Comments

Research is always helpful. The major research being done in education is being conducted in the classrooms, day by day, week by week, minute by minute. Teachers are constantly learning, on the job, through interaction with their students, what works and does not. Randomized research is helpful in giving a broad view. It is like taking a temperature with a thermometer. What works one place may work in another. What works with one group may also work for another.
One research finding that is fairly consistent is that intrinsic motivation works very well, but few curricular models are designed to motivate students intrinsically.
Medical research tends to involve clinical trials. There is an effect group and a control group. Usually the control group gets a placebo combined with established treatment. In education, this would probably have somewhat less impact in the outcome. In medicine, lives are at stake. In education, it is just learning. There is no problem with using research to seek effective practices, but the daily, ongoing research, by teachers in the field should not be overlooked.

Rigorous experimental research is appropriate for certain kinds of educational treatments/interventions, when the outcomes can be agreed upon by all interested parties as valuable (e.g. staying in school or learning to read).

However, many educational objectives have a value dimension. For instance, the NSF-funded curricula (vs. traditional math curricula) are sometimes shown to have equivalent outcomes on "fair tests." However, fair tests do not tell the entire story of what is learned with those curricula, since other topics are studied, and since the NSF-funded curricula aim for positive attitudinal/affective outcomes as well.

So, even when research demonstrates that the curricula "do no harm" on traditional measures, and provide additional positive outcomes, traditionalists oppose the use of those curricula on value-based grounds. There is also the question of who benefits--middle-class parents will vigorously oppose de-tracking in spite of the benefits to the majority of students, even if evidence demonstrates that even the best students do no worse on tests. Again, values, and perhaps implicit classism or racism enter into the picture in a way that does not happen in medicine.

My second and third criticisms do not center on a particular research model, but rather the notion that problems in education stem from a lack of rigorous evidence.

There is a resource issue, in terms of both human and financial capital. A major issue in education is what a teacher must know and do in the classroom to be effective, an issue not touched (to my knowledge) with doctors of medicine. Even supposing that one can untangle the various issues and rigorously demonstrate that certain behaviors are effective or certain knowledge of teaching is essential, where are the resources that will support teachers in adopting these behaviors or learning the essential knowledge or in recruiting those appropriately qualified individuals? For instance, while a teachers' knowledge of mathematics has a murky relationship to improved educational outcomes, it is clear that the teacher must have some fluency in mathematics in order to teach it. Money allocated to education is not likely to be as elastic as it is in medicine: When you have to pay more money for life-saving treatment, you do it; when someone else's kid will benefit from training his/her teacher, the impetus to spend money is much weaker.

Finally, for research to be effective, the American culture surrounding teaching must change. I have known and worked wih many teachers who will not change their practice in spite of evidence that such change will benefit their students. We need to rise above "this is my style" or "good teaching cannot be measured" and admit that certain things are effective and need to be done.

I am doubtful that further experimental research would have a significant positiive effect on the time- and cost-effectiveness of reading instruction. At present, there is a plethora of empirical findings obout how best to conduct this tuition. However, to a large extent, professors of education at universities across the nation refuse to inform future teachers about these data. The National Council on Teacher Quality's 2006 study, "What Education Schools Aren't Teaching about Reading and What Elementary Teachers Aren't Learning," reveals the details of that matter.

One problem with research is that it has a tendancy to seemingly come from above. Many teachers feel that a hierarchy of researchers are passing judgement and sentence upon their teaching methods. One often heard comment is, "I am with these kids every day. Who are these people to tell me what is best?" There is not so much a resistance to shange as a resistance to ivory tower proclamations.
I have noticed more parents and teachers beginning to question the value of using their children as experimental subjects. All would like to see better methods, but not necessarily at the expense of their children as educational "labrats".


The vision of medicine as a model for the improvement of teaching and education had much appeal when seeking to improve the status of teaching and teachers.

The image of the teacher as reflective practitioner and clinically analyzing each individual learner and then drawing on research to tailor instruction sounded attractive. Like most such efforts to elevate the status of something by draping it in the trappings of something else, this one seems ill-fitting for many reasons.

The analogy to medicine also fails because even if research showed that certain treatments... technologies, human tutoring, smaller class sizes... were effective, schools, unlike the medical establishment, cannot simply write a prescription and have Medicare cover the treatment.

And the analogy fails because most doctors have time to deal with individual patients and prescribe individual treatments, whereas teachers are fortunate to get five minutes away from large classes to go to the bathroom.

Human life expectancies and health have increased more by public health measures, such as clean water, sanitation, and immunizations than by wise doctors with good bedside manners. Schools need to keep telling society that they cannot overcome dreadful conditions under which so many children in our affluent country live. Perhaps the schools would serve students better by speaking truth to politicians by saying that many children without decent medical coverage and nutrition and housing during their preschool years will certainly be left behind.

Finally, the medical research analogy fails because the benefits of so many new treatments are ambiguous at best and in addition to being exceptionally expensive, often have deleterious side effects.

Perhaps better metaphors for education would be such things as community centers or summer camps or scouting or gardening.

I agree with Bob that teachers are conducting research in their classrooms all day every day. Unfortunately, this classroom-generated knowledge about teaching and learning is often ignored, or at best, marginalized because it is "just teachers" conducting the inquiry. I suggest that one of the reasons why education policy isn't working in this country is because it isn't informed by the people who actually know what works in real classrooms--the teachers. So-called "scientifically-based research" that denies instructional programs or materials to some students and calls them the "control group" is borderline unethical. The statistically significant results of this type of research are often irrelevant to the actual work going in in real classrooms. The "empirical results" that Patrick Groff refers to aren't used in colleges of ed and aren't read/applied by real teachers largely because much of it isn't designed or conducted in ways that make sense to classroom teachers and their students.

Somewhere in the midst of all the cogitation about whether the medical model, or some other, fits best for education, there are some pieces of research knowledge and best practice that need to be understood and adhered to.

As Mr. Frangione, and others, have pointed out, there is frequently an assumption that "research" is somehow being dumped on teachers by unknown ignoramuses above somewhere. Yet, few teachers are actually using research theory and carefully controlled observation in their own "classroom research." Knowing what you know is not the same as classroom research. It is unfortunate that this is so often held up as the ideal in education.

There are probably more similarities to medicine than are admitted or understood. While some may assume that a doctor only has to write a prescription and let medicare pay the bill, this is far from the truth. In fact, only recently has medicare had anything to do with paying for prescriptions. Many people lack any health coverage. Some lack means to maintain a proper diet or are so stressed by obtaining the basics of food, clothing and shelter on a regular basis that good health gets little attention. Many practitioners face these realities on a daily basis.

The ethics of providing experimental treatment or a placebo are very similar in medical research. So are the pressures to develop and market "solutions" (new drugs) that show some demonstrable impact.

Yet to return to the classroom, it is unnerving for educators to be so quick to dismiss/misuse/abuse the realm of research. In my state two separate teachers unions in two cities have recently presented "surveys" consisting of some very unscientifically collected gripes (one was gathered from comments at a single meeting consisting of less than one half of union representatives)from teachers, for which they are demanding policy responses. Certainly educators could do better.

I have struggled for years with teachers who cannot or will not write measureable objectives in an IEP. The same shows up in school improvement plans--vaguely worded improvement goals supported by whatever "strategy" appears to be handy at the moment and a measure that guarantees the ability to pick and choose whatever outcome looks best at the end of the year. Writing and testing a behavioral hypothesis for a behavior plan (and we are talking about hypothesis 101, middle grades science version, not graduate level) makes people's eyes glaze over.

I am never certain if teachers don't know, or don't want to know, or just don't want to be bothered with letting on that they know. The principles of science do have an application in the classroom, every day. As a counter example, I recall a teacher who shared why she had stopped using spelling lists in my daughter's class. First, she observed that they didn't seem to be improving spelling. Then she looked at the existing research that confirmed her finding. Then she found replacement strategies (enhancing dictionary skills, spelling patterns, etc) better calculated to have an impact. It's no coincidence that she was also very effective at teaching science.

I personally read and value research. I am constantly searching for innovation in the classroom and most, though not all of my colleagues tend to observe and try approaches to improve their teaching. It is sometimes a borow from research and sometimes it is simply instinct.
Research must be read and applied with a critical eye however. It is not enough to show that something works in randomized samples. This is indicative, but not clear evidence of effectiveness. The proof, as the old saying goes, is in the pudding. In the article under discussion, Mr. Barons uses a subtle and often overlooked strategy to make a point. It is a kind of choice in reporting that adds an air of validity to research. Towards the edn of the article, it is suggested that research funding that is currently at about 250 million ought to be doubled. To further the argument, Mr. Barons states that such an increase would amount to only about 0.06 percent of the 400 billion currently being spent on education. The facts are accurate, but left out is that the increase would bring research funding to 500 million, which would be about 0.12 percent of 400 billion. 0.12 is still a seemingly small number, but the author chose to use the smallest accurate number. Compared to 400 billion, 500 miilion seems a pretty paltry sum, but few of us have an extra 500 million laying around.
The medical studies cited showed that long established practices were perhaps not as effective as they had been thought to be. The same practices were however established after research. It often takes many years of practice to prove research findings. Perhaps that is why doctors practice medicine. Maybe teachers should consider practicing education.
My point is that teachers should become researchers. Teaching is a dynamic and ever-changing art. What works today, with this student, may not work tomorrow, with that student. That is why we are continuously learning.

When I was in graduate school, the one thing I learned was to take educational research with a grain of salt. Much of it is not worth the paper it is written on.

This is because there are so many variables involved in studying the education of a human being (after all, we're really studying the mind) that it is easy for the researcher to ignore any or many of the variables. For example, there was a "research" study from one of the state universities that "proved" younger teachers are more successful than older teachers. The researchers came to this conclusion because the test scores for the students of the younger teachers were higher. However, because I have been teaching so long, I know there are other variables involved. Here are a few of them: Younger teachers are often more desperate to keep their jobs and MIGHT engage in more test prep before administering the test. Older teachers MIGHT learn that the most important thing they can do is develop a good rapport with each child and keep the joy of learning alive so they MIGHT spend less time with test prep. Personally I don't believe the most crucial aspects of an education can even be measured by a test (healthy mind and body, love of learning, creativity, perseverance etc.). Of course I am just giving my opinion, but the point is that one test will probably not tell us that younger teachers are more effective than older teachers. There are just too many variables involved.

On the other hand, there is some educational research that has been consistent over the years. We know for example that standardized test scores (including the SAT) correlate very highly with the socio-economic status of the person taking the test. To me this means we should try to look at the home and neighborhood factors that influence a child's education. See "Monopoly and No Child Left Behind", Education Week, April 11 and "Public Rhetoric, Public Responsibility, and the Public Schools", Ed. Week, May 16. Researchers have been saying these same things for many years; yet people keep expecting teachers to bring everyone up to grade level without addressing the basic needs of all children. I guess it's a lot easier and much cheaper to demand that teachers bring everyone up to grade level by 2014 than to make sure each child has adequate housing, nutrition, safety, stimulating outside experiences and healthcare.

I do agree with others that a teacher ought to be willing to be aware of the good research and to apply it in her own classroom. As a reading teacher, I believe the research that stresses the importance of phonemic awareness in beginning reading. Almost every good reader has it and almost every poor reader does not. This knowledge helps me to offer my poor readers lots of practice in phonemic awareness each day. Sure enough, when they start to catch on to singing Bingo, Wingo, Singo, Ringo etc. they soon begin to decode cat, rat, sat mat.

In conclusion, the teacher must be a very critical reader when she reads about educational research. It isn't math.

It doea appear that Linda is correct in the assumption that it is "easier and cheaper" to expect teachers to be accountable for the "problems" with education. Standardized testing has done little, if anything to improve education in the U.S. There is very little that a testing system can do. Tests tell us how far students have moved towards achievement, but tell very little about why students achieve or do not. It is, quite frankly, wasteful in time and money to test and retest students until some magical proficiency level is reached. The concentration should be on the input, not the output. education is not simply opening the doors to schools and allowing access to classrooms. Learning is far more than books, papers and tests. It is far too complex an issue to be "reformed" by any quick fix.
Research has a place, but it is not the largest place in the practice of education. A close look at medical research will reveal that research is only a small part of that practice as well.

I agree with what's been said earlier about the difficulty of translating the medical model to education. What exactly would be the placebo in a randomized field trial in education? In medicine, members of the control group all receive the same non-treatment such as a sugar pill. How would one standardize a control treatment/experience for students, given the complexity and diversity of educational contexts across states, districts, schools, and classrooms? The only pure placebo would be providing no instruction to children in a subject, which is untenable and unethical.

I would also point out that randomized field trials are not the only form of research in medicine. Case studies and epidemiological studies are also used to understand the symptomology and prevalence of various diseases. Moreover, a lot of basic research is done in laboratories to understand the roots and mechanisms of a disease before a drug is developed to try to attack and disrupt it. Thus, the randomized field trial is the culmination of a long process of trying to understand the phenomena of a disease using a variety of research methods.

Also, there are a lot of practices within medicine that are not based on randomized field trials. My understanding is that surgical procedures are not based on randomized studies. Rather, they are a form of craft knowledge developed my surgeons and passed on.

So, if we really wanted to apply the medical model to education, we would use a plethora of research techniques. Qualitative research, observational research, and teacher research would help us develop a deeper, fine-grained understanding of certain educational processes and mechanisms. This understanding could help drive the development of educational interventions (such as curriculum materials or a professional development module), which could then be evaluated using randomized control trials.

The problem, as I see, it is not that we're using the wrong research methods and need to do more randomized trials. The problem is one of coordination and capacity. The educational field is too fragmented. No one spans the process from basic research to development of interventions to evaluation. Higher ed professors do a lot of basic research but don't do much development of interventions, in part because doing so is not rewarded in tenure decisions. Unfortunately, developing a textbook doesn't count for much in a research university, so people focus on peer-reviewed journal articles. There are a lot of other perverse incentives in academia. Curriculum materials are developed by publishing companies and may or may not be based on the latest research, and are subject to political compromises. And there is little money to conduct large scale evaluations of interventions.

What's different in the medical field is that pharmaceutical companies because of the structure of the industry and the large profit motive have an incentive to integrate the process. They use basic research funded by the public (and increasing by themselves) to develop drugs and then subject them to randomized trials, in the hopes of arriving at a blockbuster drug.

I'm worried that there will be rush to study interventions using randomized study designs without a clear understanding of why we believe such an intervention might work. Without this prerequisite knowledge we might: (1) be wasting time and resources on a fishing expedition, or (2) even if we find significant findings, we might not know how to translate them into practice (since we don't know why the intervention works and what aspects of it are the critical ones and what aspects are merely incidental).

I think most of us can agree that there is a pressing need for quality research in the field of education. Bob makes an excellent point when he states that teachers are often the best source of classroom-based research. However, as Gail points out, such data is regarded as inherently flawed because the researchers are "just" teachers. However, in the medical profession, research is conducted by "just" doctors all the time. Research in the prevailing medical journals is published not only by doctors whose primary field is research, but by doctors who are practicing in their field as well.

One of the ways to alleviate this problem of research being conducted by "just" teachers is to do something radical: PROFESSIONALIZE the teaching profession. Teachers in this country are not treated as professionals. Instead, they are treated as laborers. Teachers are paid by the hour, are represented by unions, and have strict guidelines covering what jobs that they can and cannot do on school campuses. Teachers are universally blamed by politicians as the root cause of all of the educational ills that befall this country (though, as Linda points out, socioeconomic status and home environment often have more to do with student success or lack thereof than any educational program that can be put in place).

Instead of demeaning teachers, policymakers should treat them like the professionals they are. Teachers are highly educated; I know very few teachers who do not possess a masters' degree. According to the 2000 census, this places most teachers in the top 10% of educational attainment in this country (8.9% having attained graduate or professional degrees by the 2000 census). Teachers must undergo lengthy training, both in university and on-the-job training, and are expected to keep up their training to maintain their certification. This practice is also alive and well in the fields of medicine and law, two very highly regarded professions.

In order to professionalize the teacher corps in this country, a few things need to happen:
1) Pay teachers as professionals. Many teachers must work second jobs or summer jobs to make ends meet (so much for having summer off... I've never had a summer off in my five years in the classroom!). Don't just take my work for it... check out "Teachers Have it Easy" by Moulthrop, Calegari, and Eggers. When was the last time that your doctor or lawyer had to work a summer job?

2) Treat teachers like professionals, not like babysitters. Teachers are highly educated and highly trained. They are not "just" teachers. Give them the benefit of the doubt.

3) Restructure the schooling system to give teachers time to become better teachers. American teachers spend vastly more time in contact with their students than do teachers in other industrialized countries. While teacher contact time is nice, when are teachers supposed to reflect on their practice, design research-based lessons, and evaluate student work? Teachers in other western industrialized countries spend about half as much time with students as we do, and spend the rest of the time collaborating with other teachers, planning, and evaluating student work. No wonder they are kicking our butts at all those international tests...

The time has come to treat teachers as professionals, and to revamp the educational system in this country. Until then, the "research" is going to come to the same conclusion: American students aren't achieving as well as they should, because their educational system is broken. The first step is revolutionizing the way we treat teachers in this country. Then we change the way we teach (and maybe even what we teach). Then, we change the way we research.

Don't get me started on changing the way we take care of people's basic needs...

This-or-that works better compared to what, is probably the question. We usually want to evaluate some new way, in relation to the effectiveness of the traditional way.

This appears to assume a process of innovation, introducing 'new ways'; new forms of school and schooling.

I'm assuming the innovators need to be free to try these new ways/forms, in real schools.

So these new forms themselves cannot be subject to a test of 'proven' through research: Correct? If I am not right about this, please tell me.

This would also suggest that to get more from research we should be doing more innovation. Correct?

Comments are now closed for this post.

Advertisement

Recent Comments

  • Ted Kolderie Education|Evolving: This-or-that works better compared to what, is probably the question. read more
  • Bryan, Educator/Doctoral Student: I think most of us can agree that there is read more
  • Ed/Professor of Educational Administration: I agree with what's been said earlier about the difficulty read more
  • Bob Frangione, Educator/Parent: It doea appear that Linda is correct in the assumption read more
  • Linda/Teacher: When I was in graduate school, the one thing I read more

Archives

Categories

Technorati

Technorati search

» Blogs that link here

Pages