MET Report Could Influence District Human Capital Decisions
Earlier this week, the Bill & Melinda Gates Foundation released "Ensuring Fair and Reliable Measures of Effective Teaching," the final report of the organization's three-year long Measures of Effective Teaching (MET) project. I would encourage all teachers, education leaders, and talent managers to read the report as it has huge implications for the future of teacher evaluation and other human capital decisions in states and districts across the country.
The MET project is a partnership of more than 3,000 public school teachers (grades 4 through 8, as well as high school grade 9 English, Biology, and Algebra I) who voluntarily opened up their classrooms to researchers. The study looked at value-added, evaluation, and student surveys with the purpose of investigating "better ways to identify and develop effective teaching" as well as "help teachers and school systems close the gap between their expectations for effective teaching and what is actually happening in classrooms." Participating districts included Denver Public Schools, Dallas Independent School District, Memphis Public Schools, Pittsburgh Public Schools, New York City Schools, Charlotte-Mecklenburg Schools, and Hillsborough County Public Schools.
As I noted in a previous post on the MET project, "Teacher Evaluations & MET Project Findings," many well-known policy and research groups were involved in this study, including the American Institutes for Research, Educational Testing Services, National Board for Professional Teaching, National Math and Science Initiative, New Teacher Center, RAND, Teachscape, Westat, and many other organizations, as well as several universities, including Chicago, Dartmouth, Harvard, Michigan, Rutgers, Stanford, Texas, Virginia, Washington, and more.
Final Report Findings
Following are key findings and excerpts from the final MET report which impact teachers, building-level leaders, district and other administrators, as well human resources staff and processes.
• Great teaching CAN be measured:"The data show that we can identify groups of teachers who are more effective in helping students learn. Moreover, the magnitude of the achievement gains that teachers generated was consistent with expectations. In addition, we found that more effective teachers not only caused students to perform better on state tests, but they also caused students to score higher on other, more cognitively challenging assessments in math and English." (Pages 4 -5)
• Teachers need meaningful feedback to grow, so provide it: "Student perception surveys and classroom observations can provide meaningful feedback to teachers. They also can help system leaders prioritize their investments in professional development to target the biggest gaps between teachers' actual practice and the expectations for effective teaching." (Page 20)
• Observations should be done by multiple reviewers, multiple times: Specifically, the report notes that shorter, more frequent observations from two or more observers per teacher provide a more reliable snap-shot of true teacher performance, rather than one individual performing a single, longer observation.
• Building processes that increase trust and fairness will result in better data: This shouldn't surprise anyone. Having detailed, communicated processes when it comes to evaluations and student surveys, as well as rigorous training and testing of observers against a set of standards or expectations increases data quality. Further, the report notes that due to the advances in video technology, utilizing such methods to provide feedback and train observers is promising.
• Surveying students? Ensure Confidentiality: While student surveys are a highly-debated topic, Gates used them to gauge teacher effectiveness. Specifically, "student perception surveys that assess key characteristics of the classroom environment, including supportiveness, challenge, and order." (Page 3) The report notes that student survey data becomes more reliable when students feel that they are able to provide anonymous feedback.
• Utilize multiple measures when building teacher evaluation or performance index formulas: "Compared with schemes that heavily weight one measure, those that assign 33 percent to 50 percent of the weight to student achievement gains achieve more consistency, avoid the risk of encouraging too narrow a focus on any one aspect of teaching, and can support a broader range of learning objectives than measured by a single test." (Page 20)
It is important to recognize the differing views of education leaders and organizations about the MET report's findings. Stephen Sawchuk notes a "mixed reception" to the report in his recent Education Week article.
Randi Weingarten, president of the American Federation of Teachers, said in a statement, "The MET findings reinforce the importance of evaluating teachers based on a balance of multiple measures of teaching effectiveness, in contrast to the limitations of focusing on student test scores, value-added scores, or any other single measure."
At the same time, Arkansas professor, Jay P. Greene, who was quoted in Sawchuk's article (as well as in the New York Times), was not as supportive of the report. In a blog post, "How Gates Spins its Research," Greene argues that Gates spun the research to make the case that multiple measures are better than one. Citing information from the report, he explains, "If both the lay-version and technical reports had always shown how little test scores are improved by adding student surveys and classroom observations, it would be plain that test scores alone are just about as good as multiple measures." He followed this up with another post, "Understanding the Gates Foundation's Measuring Effective Teachers Project."
Teacher evaluation and the use of measures to gauge educator effectiveness is a hot topic in states and districts across the country. I will be interested to see how the MET report influences federal and state legislation as well as district policies, particularly around talent management, in the years to come.
For more information on human capital and talent management in education, you call follow me on Twitter: @EmilyDouglasHC