My Research Is Better Than Your Experience
During the first class of a doctoral-level course sequence on education research, my professor mentioned that he wished researchers would refrain from writing policy implications when reporting on their research findings. Just do the study, tell 'em how you did it and what you found, he said. The policy implications section is invariably the least defensible part of any research paper, the place where researchers' inherent biases emerge. Let your readers decide--or argue about--what the results mean, in terms of policy-making and practice applications.
Folks in my Ed Policy cohort were appalled--why would you go to the trouble of investigating an important research question if you couldn't follow through with policy suggestions based on the Truth and Wisdom your study uncovered? The more studies we read and analyzed, however, the more his observation resonated with me.
It's remarkable how many well-done research studies present credible evidence, then go off on an implications bender. Even when evidence is shaded or questionable or very limited, the boldness of the associated policy recommendations would often make your head spin. If you were a practitioner, that is, thinking about how this would play out in an actual school.
Here's an example of what I mean, clipped from Mathematica's recent study on the effectiveness of TFA math (note: not literacy) teachers:
Although TFA is often criticized for the fact that its teachers make only a two-year commitment to teaching, the findings suggest that over the long term, continuing to fill a position with TFA teachers who depart after a few years would lead to higher student math achievement than filling the same position with a non-TFA teacher who would remain in the position and accumulate more teaching experience.
In what parallel universe--given what we know about the financial negatives of hiring and inducting new teachers, the damage to teacher collaboration caused by constant staff churn, the point at which any teacher hits her instructional stride, and so on--would we imply that it's ever OK to fill any school department with revolving temps? That's not how responsible school leaders build a focused academic climate in a building, or what parents are willing to invest in or support.
And then, there's the apparently never-ending research debate over learning styles, which bubbled back up again this week in Scientific American, with an article exploring whether addressing learning styles in instruction was a "bogus idea." The usual wrangling ensued, with dueling experts splitting semantic hairs over whether a cognitive strength or preference (which, everyone seems to agree, do exist) can be called a learning style. Psychologists, professors and a pediatric optometrist weigh in, citing research evidence or lack thereof, turning the piece into a "my research is better than yours" battle.
Notice anybody missing in this discussion?
On the same day the Scientific American article was published, Sarah Darer Littman posted a thoughtful blog from her practitioner perspective, about how she alters her approach to individual learners, in teaching writing. She questions the idea that there is no measurable value in teaching to students' innate preferences, or "styles," using an example from her own practice. For her trouble, she gets pushback about "scientific" research, trumping her own reflections. Littman says: "what informs my teaching most is parenting a child who thinks, processes and learns very differently than I do."
Welcome to the club, Sarah. I have produced several blogs (here, here and here) exploring my own years-long investigation of learning styles in teaching music, concluding that we privilege certain instructional techniques and learning modes in music performance classes.
In doing so, we often leave out kids who could really enjoy and benefit cognitively and socially from being part of a school musical group or activity. Simply by allowing students to learn to play a musical instrument by listening and imitation, rather than beginning with note-reading, I engaged lots of students with identified learning disabilities in the band program.
Were those students "auditory learners?" I don't know. And I don't think it matters. I found a way to teach them, playing to their strengths. And that was good for everyone in the music program.
That's the sticking point in utilizing research findings to make grand pronouncements about how schools should operate, or how teachers should teach. We leave out the most important voice, that of the people doing the actual work, and the ideas they find useful.