Circular Reasoning at the Gates: Education Nation off to a Confusing Start
Last September NBC brought us the first Education Nation, developed in coordination with the release of the pro-charter documentary, Waiting For Superman. The network ran into a few bumps in the road, catching flak when it was pointed out that panels were loaded with "superheroes" like Michelle Rhee, and critical voices like Diane Ravitch, and those of classroom teachers, were largely absent.
This year, NBC has made an effort to be a bit more balanced and inclusive of teachers voices, and the Teacher Town Hall yesterday made a start in that direction.
On a stage dominated by the largest golden hood ornament I have ever seen, Brian Williams interviewed mostly teachers, while Tamron Hall roamed about the audience taking comments from the crowd.
The comments from the teachers present are worth a listen, but my mind kept dwelling on the interview with Melinda Gates. First, here are some of the things Brian Williams said about Mrs. Gates and her husband.
At the top of the show, we were told:
We're also going to be joined by Melinda Gates, co-chair of the Bill and Melinda Gates Foundation. Gates Foundation, one of the sponsors of this event, and the largest single funder of education anywhere in the world. It's their facts that we're going to be referring to often to help along our conversation.
Then, in his introduction of Melinda Gates, Brian Williams said:
You could refer to our guest as the top funder of education in the world. A partner and sponsor of this year's gathering. Also spending half a billion dollars to devise a way figure out what makes a great teacher, what makes them most effective. The estimates are the Gates Foundation has already spent, obviously a record for any education spending, spent or committed to spending five to seven billion dollars.
But I want to focus on what Mrs. Gates said, because there is something deeply disturbing about the way the issues have been framed. And since this foundation is, according to Brian Williams, the source for the very facts that are guiding this conversation, it seems crucial to understand the thinking that is behind their work. Please review her thoughts, and see what you think of the reasoning that is at work.
Brian Williams asks:
You and your husband have always said this all comes back to a single relationship a student and a teacher. What have you learned about what makes a great teacher?
Melinda Gates responds:
Everybody says 'you can't just look at test scores at the end of the year, because there are so many factors, there's poverty and other things that go into this.' But nobody had done the research to say 'how do we know that a teacher's making a difference in a student's life?' So we set out to do this enormous piece of research. Three thousand teachers signed up in six different districts. We videoed the teachers, and we said 'at the end of the day, what is predictive of great teaching? What besides that test score?' And it turned out a teacher who's good one year is good usually in the second year. It turned out you could look at the test scores and see in terms of value added, how had they moved kids up in the system. But then you could also look at student perceptions. It turned out that student perceptions of a teacher were also predictive of how they would do at the end of the year and whether they learned all that material.
How do you keep that from becoming a popularity contest?
We learned you have to have multiple measures of what make a great teacher. Right now teachers are observed by their principals at regular intervals. We need to have peer observations. But we need to know that the tool that we're using -- there are ten different tools for peer observations. But which ones actually predict whether the students learned the material at the end of the year? So we need to test the peer observations, and the principal observations, and we need to look at the scores at the end of the year, and we need to look at the student data. When you ask the students did you have an effective teacher, you ask specific questions, 'did the teacher help you when you didn't understand the homework, or what you missed on your homework? Did they go help you learn that? Did the teacher get a sense of when he or she didn't explain the information well, and help get your class on track? Did your teacher manage the classroom well? It turns out there are about six questions you can ask the students - not 'did you like the teacher,' but what they did in the classroom that actually measures and correlates to whether the test scores got better at the end of the year.
Do you notice what is bothering me? Mrs. Gates begins by acknowledging that good teaching cannot be reduced to a test score - or at least that this is often said. She then asserts that the half billion dollars they have spent on research in this area have uncovered a number of things that can be measured that allow us to predict which teachers will have the highest test scores. A great teacher is defined over and over again as one who made sure students "learned the material at the end of the year."
If you look closely at how she describes peer observations, the method at work is even clearer. Teachers tend to support peer observation, because it can be a valuable basis for collaboration, which yields many benefits to us beyond possible test score gains. But what does Melinda Gates say about it? It can be worthwhile, BUT: only the models of peer observation that have been proven to raise test scores should be used. And presumably we can count on the Gates Foundation to provide us with that information.
In spite of all the billions they have spent, it appears that the Gates Foundation is laboring under the same logical fallacy that doomed No Child Left Behind. In a way which employs circular reasoning, they have defined great teaching as that which results in the most gains on end of year tests, and then spent millions of dollars identifying indicators of teaching that will yield the best scores.
The most deceptive strategy is how they then try to pretend that these indicators are "multiple measures" of good teaching. In fact, these are simply indicators of teaching practices associated with higher test scores. In spite of Mrs. Gates' feint at the opening of her response, everything she describes, all these things that supposedly go beyond test scores - peer observations, student perceptions - are only deemed valid insofar as they are correlated with higher test scores.
Melinda Gates begins with the question "How do we know a teacher's making a difference in a student's life?" That is an excellent and complex question. However, when we look at her answer, we find she commits the logical fallacy known as "begging the question." One begs the question when one assumes something is true, when that is actually a part of what must be proven.
The question she begs is "what defines great teaching?" This is not answered by finding teaching methods associated with higher test scores. This question remains hanging over the entire school reform enterprise. Until we answer that question, we are devising complex mechanisms to elevate test scores assuming this will improve students' lives, when this is manifestly unproven. In fact, I would argue that many of the strategies used to boost scores are actually harmful to our students. And many dimensions of great teaching are not reflected in test scores -- and are systematically undermined when educators are held ever more "accountable" for these scores.
This episode should remind us of the crucial need to teach critical thinking in our schools - and apply such thinking to the dilemmas we face.
The other thing that was rather disturbing was the omnipresence of the Gates Foundation's largesse. Towards the end of the show, Brian Williams offered this advice to viewers:
This is a couple who have decided to give away their fortune. I heard two educators earlier today, one said to the other, "they never set out to do anything other than put money into education and help kids." So thanks to our audience for being mindful of that.
There was some pushback, however, and NBC deserves some credit for giving space for some differing views. New Haven teacher Matt Presser was one of the winners of an essay contest, and he offered his thoughts:
Too often school reform is something that is happening to our students as opposed to with them or for them, and so many decisions are being made by people in board rooms, people in the White House, when the real people who know what our students need are the people here today, the people in our classrooms every day.
This must have seemed to be a bit ungrateful to Brian Williams, because he then asked,
We just had Mrs. Gates here. This is a guy, I think the Forbes latest figure is $60 billion...here's the Gates family, spending upwards of $7 billion so far, haven't broken a sweat yet, trying to talk to you guys, ask you questions, including students, asking questions about what's working, what's not working. Do you support their efforts? Do you think it's money well spent?
Matt Presser replied,
I think it's a shame that we have to rely on philanthropy to support our schools, to make up for an educational debt that has accrued for generations. I think certain communities, especially in urban areas, have been neglected by education for so many years, we have so much to make up for - not just in education, but in housing policy and job discrimination. In so many areas across the country, that even those efforts to get more money into our schools, there needs to be more a wholistic approach, instead of just something that is thrown at our schools.
But perhaps the most potent counterweight to the Gates approach was offered by teacher John Hunter. He said,
My first job interview, I asked the supervisor, what should I do? She said "What do you want to do?" As a teacher, to be given that kind of open space, that kind of mandate-less position to be in where you can create out of the emptiness, it allowed me to create that kind of template for my students, where I could ask them, "what would YOU like to do today? What is your passion? What drives you?" If the students have the interest and you build towards that, then they can come with more passion for learning.
He took advantage of this latitude to create a now-famous eight-week long interactive game where his students are challenged to solve world problems. Was this great teaching? Do we have to wait until we see how his students performed on the end-of-year standardized tests to find out?
What do you think? Does the Gates Foundation's research into great teaching beg the question by relying so heavily on test scores? How does this affect the direction of education reform?