Hi, everyone! I've been traveling in Asia for the last month, which is why you haven't heard a peep out of me. I started in Bangkok, Thailand, and then traveled eastward into Cambodia and then north through Vietnam. It was interesting and beautiful, and a really exciting change from New York City. Also, I had a chance to talk to some other teachers from England and Ireland, which I will discuss in later blog posts. (Apparently, I'm not the only teacher who thought to spend summer vacation backpacking through Indochina.) Now I'm back, re-immersing myself into all the education issues that can seem blissfully irrelevant during July, but won't be, come September.
In a post I wrote a month ago, I talked about how my students' high pass rate on the New York State Regents exam in English--while it made me look good, when viewed in isolation--was in fact a superficial indicator of my proficiency as a teacher, and really just showed that my class was better prepared than their predecessors (who had a comparatively lower class pass rate, not just on my exam, but on all their exams.) My hunch that I couldn't start patting myself on the back was confirmed when scientists from the Human Resources Research Organization (HumRRO), who specialize in Industrial-Organizational Psychology, sent me a series of informative and interesting articles about the problems of evaluating teachers based on their students' test scores. I'll try to distill the main points of these articles here:
Essentially, when teachers are being evaluated, we talk about measuring their "performance" and their "effectiveness." Most people use these two terms interchangeably, however, they're not the same. Performance refers to what the teacher actively is and does, day to day, which is directly observable: Having good content knowledge, interacting positively with students, creating diverse and interesting lessons, exerting classroom management, etc. Effectiveness, as used here, refers to the test scores--ideally, high student test scores may be the results of strong teacher performance. Or, they may not be. Therein lies the problem with viewing the two qualities interchangeably.
Drs. Rod McCloy and Andrea Sinclair give the example of two sunglasses salespeople. One is a high performer--has a great sales pitch, knows about the products, etc. The other is highly effective--in that, under his watch, people buy a lot of sunglasses. However, the highly effective salesperson works in a sunny climate like Las Vegas, while the high performer works in Seattle. Who sells more sunglasses? The highly effective salesman, of course; great product knowledge and strong sales techniques can only go so far towards trumping outside factors such as a total lack of sunshine (and thus low demand for a product like sunglasses.) High efficacy may be correlated, but does not necessarily present a direct cause-effect link, with performance.
The same logic can be applied when talking about teachers. In 2012, when my students took the Regents, only 80% passed. In 2011 and 2013, 97% passed. Was my performance inferior in the 2012 year? Unlikely, since I used almost the same units for all three cohorts, remained the same person, and the test itself was virtually unchanged during that time. More likely, there were outside factors that impeded my effectiveness--factors for which (at least in some cases) I could not directly control: Maybe it rained the day of the test, and students showed up late and frazzled. Maybe individual students did not eat breakfast. Maybe, as I posited, that group itself was less prepared for 10th grade (as evinced by the fact that all their exam scores, across the board, were lower than those of their peers in the classes above and below them at the 10th grade benchmark.) Certainly, one might look at my performance in the 2012 year and compare it to 2011 and 2013 in order to try and determine how I might be more effective; however, that's not to say that effectiveness and performance are the same thing, or even interchangeable.
This distinction has important implications for evaluating teachers. While students' standardized test scores will likely be used as a component of a given teacher's evaluation, to use that is the only or largest component is misleading--and results in evaluating teachers based on too many factors that are outside of their control, while ignoring important direct indicators of proficiency (quality of lessons, classroom management, rapport with students, etc., as well as consideration of the school and system within which the teacher works.) This is not to say standardized tests are valueless, as Drs. McCloy and Sinclair clearly point out; rather, it is to say that we need to take many factors into consideration when evaluating a teacher, rather than trying to boiling the process down into some superficial numeric representation of pedagogic proficiency.
I'd like to talk more about this in subsequent blogs, and I hope that I've given a helpful (and correct) initial overview of this topic (which I find fascinating, since in my heart of hearts I like nothing better than "geeking out" over social sciences.) That's all for now, but more to come in the next few days.