“Don’t think about elephants,” skoolboy’s father used to joke, long before George Lakoff’s manifesto with a similar name. The joke, of course, is that by trying not to think about elephants, all that you can think about is elephants. The harder I tried not to think about elephants, the more I thought about them.
The New York City Department of Education has its own variation. This month, the DOE is sending Teacher Data Reports, which purport to estimate the effect of individual teachers in grades 4-8 on students’ test scores, to school principals, who will then distribute the reports to their teachers after the principals have been trained. “The Teacher Data Reports are not to be used for evaluation purposes,” wrote Chancellor Joel Klein and UFT President Randi Weingarten in an October letter to teachers. “That is, they won’t be used in tenure determinations or the annual rating process. Administrators will be specifically directed accordingly.” Similarly, the Frequently Asked Questions section of the DOE’s Teacher Data Tool Kit website poses the question “How can you be sure that principals won’t use the Teacher Data Reports to evaluate teachers?” The response: “Principals have been and will continue to be explicitly instructed not to use Teacher Data Reports to evaluate their teachers. The DOE has standard processes in schools for teachers to raise issues or concerns.”
And yet. From the Frequently Asked Questions on the DOE’s Teacher Data Toolkit website: “By isolating individual teachers’ contributions to student progress, the Teacher Data Reports provide valuable information to school leaders and teachers about where to focus instructional improvement efforts. …Teacher Data Reports provide information about how individual teachers’ efforts influence student learning … A sophisticated multivariate regression analysis based on NYC data from 1999-2008 determined how much to weigh each factor [to calculate students’ predicted gains] … A panel of technical experts has approved the DOE’s value-added methodology. The DOE’s model has met recognized standards for demonstrating validity and reliability. Teachers’ value-added scores from the model are positively correlated with both School Progress Report scores and principals’ perceptions of teachers’ effectiveness, as measured by a research study conducted during the pilot of this initiative.”
In other words: The Teacher Data Reports rely on sophisticated statistical techniques that are valid, reliable and approved by experts, and they isolate an individual teacher’s contributions to student learning. But, you principals who are under tremendous pressure to increase test scores or face losing your jobs, don’t you dare think about using these Teacher Data Reports to evaluate teachers.
Don’t think about elephants.