The massive Measures of Effective Teaching Project is finding that teacher effectiveness assessments similar to those used in some district value-added systems aren't good at showing which differences are important between the most and least effective educators, and often totally misunderstand the "messy middle" that most teachers occupy. Yet the project's latest findings suggest more nuanced teacher tests, multiple classroom observations and even student feedback can all create a better picture of what effective teaching looks like.
Researchers dug into the latest wave of findings from the study of more than 3,000 classes for a standing-room-only ballroom at the American Educational Research Association's annual conference here on Saturday.
"The beauty of multiple measures isn't that there are more of them—more can be more confusing—these need to be alligned to the outcomes we care about," said Steve Cantrell, who oversees the MET project for the Bill & Melinda Gates Foundation. Existing teacher evaluation systems often use indicators that are not effective at guaging student achievement, and moreover that lump teachers into too-simplistic categories.
"The middle is a lot messier than a lot of state policies would lead us to believe," Cantrell said. "Teachers don't fall neatly into quartiles. Based on the practice data, if I look at the quartiles, all that separates the 25th and 75th on a class (observation) instrument is .68—less than 10 percent of the scale distribution. In a lot of systems, the 75th percentile teacher is considered a leader and the 25th percentile considered a laggard. ...This would suggest they're a lot closer than being off by two categories."
(A note of disclosure: the Gates foundation also supports some coverage of education innovation and industry in Education Week.)
Moreover, Cantrell said firing the lowest-performing quarter of teachers wouldn't improve teachers' general instruction. Researchers found differences in classroom practices between effective and ineffective teachers mostly in classroom management and behavior. Based on more than 24,000 lesson observations, the project found "Classroom practice could be described as orderly but unambitious," Cantrell said. "The average needs to be moved pretty sharply to the right."
Is it possible to test whether a teacher will use the best instructional practices, as opposed to only average ones? Drew Gitomer, education chair at Rutgers Graduate School of Education in New Brunswick, N.J., thinks so. He's working with MET to develop better teacher content assessments, which gauge "whether teachers understand how to phrase questions and examples that will address potential student misconceptions about the content."
For example, one question asks teachers to determine which of several examples of simple exponents would be the least useful in a math class; the question looks for the teacher to understand that three cubed would be a better example than two squared, because the latter would give students the same answer, 4, if they simply multiply the number by its exponent. Three cubed, if solved incorrectly, would reveal a student's misunderstanding of the procedure.
Gitomer conducted 90-minute interviews with 60 teachers who scored in the top and bottom 25 percent of teachers on that test. He found that the lowest-performing teachers often had weak reasoning for their instructional decisions, losing track of the larger purpose behind a lesson and often providing no justification for an approach beyond personal preference. Strong teachers, by contrast, tended to use questions to look at larger classes of problems, and could describe how their approach supported their students' learning.
Student observations may be the real key to identifying what works in teaching, according to Ron F. Ferguson, a senior lecturer in education and public policy at Harvard University in Cambridge, Mass. He analyzed surveys from 2,985 MET classes with at least five responding students each, and compared students' achievement with their observations of "seven Cs" of teaching practice. They focus on whether a teacher:
• Cares about students;
• Captivates them by showing learning is relevant;
• Confers with students to show their ideas are welcomed and respected;
• Clarifies lessons so knowledge seems feasible;
• Consolidates knowledge so lessons are connected and integrated;
• Controls behavior so students stay on task; and
• Challenges students to achieve.
Educators often think caring is the most critical practice for student achievement, Ferguson said, but actually the practices most highly correlated with high achievement were, in order:
• Control, in which students reported treating the teacher with respect, that their class behaved, stayed busy and didn't waste time;
• Challenge, in which students reported that they "learn a lot every day" and "learn to correct our mistakes," and, finally,
• Clarify, in which students noted that their teacher explains difficult things clearly.
Teachers who were in the top quartile in terms of the numbers of students reporting that they practiced the seven Cs, Ferguson said, had students achieving a half year of learning more than students of teachers in the bottom quartile of the seven Cs.