« Study Says N.Y.C. Charters Help Disadvantaged Pupils Catch Up | Main | Post Editorial Says Study 'Demolishes' Anti-Charter Debates »

Making Education Policy: Is Research in the Mix?

| 1 Comment

How much weight do education leaders give to research when they make decisions about schooling? Apparently, not much, according to a new report.

Released earlier this week, the report's findings are drawn from six conversations held around the country over the last year or so with focus groups made up of congressional staff members, state legislators, state and local school superintendents, school board members, and the like.

Asked to discuss what influences education-related decision making in their arenas, these folks didn't mention any pathbreaking studies in education. They cited public opinion, mandates from political higher-ups, newspaper articles, trusted colleagues, local data, and information from professional organizations and conferences. As one of the congressional staffers in the mix put it:

"It is what it is. I would certainly say the anecdotal or the real-life experiences that staff or members face are probably always going to trump research."

When the participants did talk about research, the report says, they expressed skepticism and talked about the limitations of studies. Report authors Steven R. Nelson, James C. Leffler, and Barbara A. Hansen write:

"It was a common perception of the participants that research could be shaped to say anything, that one piece of research often conflicts with another, and that much research is not timely for users' needs."

I'm struck by the difference between these findings and those from a national public-opinion poll that I wrote about last month. In that survey, which was published in Education Next, researchers found that just adding the phrase "research shows" to a question led to double-digit increases in public-approval ratings of specific education initiatives, such as charter schools. Could it be that the public puts more stock in research findings than its leaders do?

The new report does offer one bright spot: It highlights the potential role that trusted "intermediaries" can play as a conduit for research. Education leaders already turn to research organizations, professional groups, and coalitions—and trusted individuals within those groups—for research information, but the report contends that even more could be done to help education leaders apply research knowledge to local situations.

And, no, no one specifically mentioned "Inside School Research."

This report was put together by Education Northwest, the Portland, Ore., research group formerly known as the Northwest Regional Education Laboratory. The William T. Grant Foundation of New York City commissioned the study. (Not coincidentally, the foundation is headed by Robert Granger, the former president of the national board that advises the U.S. Department of Education on its research operations.) Look for the findings to inform the foundation's new research initiative on exploring and expanding the use of education research.

1 Comment

As an educator there are a couple of reasons for this skepticism...

Many of us recognize the limitations of social science as a tool (this is not the same as saying it should never be used). There are lots of things that can skew results because it's very difficult to narrow down all the inputs in a "real" classroom, and if it's done in a lab setting, it may not be replicable in the complex environment of a real classroom. A recent NY Times piece on Tools of the Mind shows an example of the difficulty in teasing out what elements are effective when a panoply of interventions are used.

I liked this analysis of the recent charter study in New York City and the point it makes about the impossibility of a "placebo" in this type of study.

In summary, a social science study will never be the same as a science study

Then, there are has been a lot of lousy research promulgated in recent years in the quest for "scientifically" proven instructional methods. Most of my experience with that has centered around Reading First, of which you will not find any meeting of minds at this point. If you'd like more details, I'm happy to provide them, but I'm guessing you and your readers don't need a recounting of the reading wars.

I don't think we can progress in our endeavors without research and effectiveness studies, but I think we also need to have a healthy skepticism about both what we are doing based on our own limited experience, and what studies say we should be doing. Skepticism doesn't mean ignoring studies, but asking hard questions, and if they come up lacking, ignoring them, and if they hold water, considering changing your practices. That's why I've added this blog to my reader ;-).

Comments are now closed for this post.

Follow This Blog


Most Viewed on Education Week



Recent Comments

  • A. Mercer: As an educator there are a couple of reasons for read more