skoolboy went to a session Thursday that was billed as about all things teachers -- mobility, retention, etc. But the session was a bait-and-switch; three CALDER (National Center for Longitudinal Data in Educational Research) papers, only two of which were about teachers. Tim Sass led off with a paper on charter high school effects on high school graduation and college attendance in Chicago and the state of Florida. Using eighth grade test scores and demographic variables as controls, and studying students who attended charter middle schools to control for selection bias, Sass and his colleagues found that students who attended charter high schools were 11 to 14 percentage points more likely to graduate from high school, and 10 to 13 percentage points more likely to attend college, than similar students who did not attend charter high schools. He concluded that expanding school choice at the high school level may be part of an effective policy to reduce high school dropout rates and promote college attendance.
Sunny Ladd presented a paper coauthored by Charlie Clotfelter and Jake Vigdor on high school teacher credentials and student achievement. Examining North Carolina end-of-course tests in English I, Algebra I, biology, geometry, and ELP (social studies), Ladd modeled achievement as a function of teacher credentials and characteristics, classroom characteristics, and student fixed effects. Students of teachers who entered via lateral entry rather than a regular license had lower test scores, whereas students with more experienced teachers and National Board certified teachers had higher test scores. Certification in the subject taught enhanced test scores by .08 standard deviations -- a sizeable amount, given that low SES black students scored .12 standard deviations below other students. Ladd found that teacher credentials explain 1/5 to 1/3 of the overall variation in teacher quality, and that teacher credentials are distributed unevenly across schools, with black students and students in high-poverty schools less likely to have highly-qualified teachers. Thus, racial differences in access to teacher credentials contributes to the black-white achievement gap.
Jane Hannaway reported on a study of Teach For America effects on high school math and science outcomes in North Carolina. (Basically the same data that Ladd used.) Estimating a cross-subject student fixed effects model, Hannaway found that students of TFA teachers performed better than students of several different comparison groups of teachers. At least in high school, she concluded, there is a greater payoff to teacher selection than to teacher retention.
Dan Goldhaber, discussing the papers, raised questions about the generalizability of the findings, and argued that the question that policymakers are likely to ask -- “What kind of a bet am I making?” in picking a policy alternative -- would best be addressed by a distribution of likely outcomes, not a point estimate of the average effect. A number of other thoughtful comments.
These are all skilled researchers, who analyzed their data with great care. And yet I came away disappointed in two respects. First, these presentations were largely atheoretical. They answered a set of “what works?” questions, but didn’t yield much in the way of insights about mechanisms. Second, the two North Carolina papers relied on end-of-course test scores, but I was dismayed that Ladd and Hannaway didn’t really know very much about the tests. One of the challenges in large-scale longitudinal data analysis is that just getting the data in shape to analyze is a big deal. But tests have psychometric properties, and no one in the room knew very much about them -- or about what the history and details of teacher certification requirements in North Carolina was. Since these were central concerns in the North Carolina papers, I left uneasy.