One of the thornier problems in education research is figuring out how to measure classroom instruction when the researcher or a camera can’t be around to record what’s really going on. If you give teachers a survey and ask them report on what they did in class, they might be inclined, just as any of us would, to make themselves look better. And students, well, who knows how accurate they are?
A new nationally representative study confirms that teachers and students do indeed disagree when it comes to describing the lessons taking place in the same middle school math classroom—but not on everything and not as much as you might think.
Writing in the March issue of Educational Policy, researchers Laura M. Desimone, Thomas M. Smith, and David E. Frisvold found, for instance, that teacher and student reports converge when it comes to reporting how often they used textbooks and calculators. Teachers are more likely, though, to “over-report” time spent on discussion, writing, and group work—the kinds of things, in other words, that groups such as the National Council of Teachers of Mathematics say they ought to be doing. But the differences, while statistically significant, were not large, according to the study.
Accounts also differ among students in the same classes. Female students, successful students, advanced-math students, and pupils whose parents were more highly educated were all more likely to agree with teachers’ reports. The researchers also found that, when they controlled for some standard characteristics of students, such as race, socioeconomic status, and achievement, they could narrow the gap some between teacher and student responses.
So who’s right? The study doesn’t say. Because the data came from National Assessment of Educational Progress tests in math, the researchers weren’t able to document what really went down in those classes.
They say the the bottom line “is that caution should be exercised when using either teacher or student reports of instruction.” At the very least, they added, studies ought to take student background variables into account or triangulate their results with those from an independent observer.