If a teacher gets intensive and well-designed professional development in a particular subject like math, you might imagine the opportunity is bound to pay dividends in the classroom.
But don’t bet on it, a new, federally funded study suggests.
The study on the cumulative impact of providing 7th grade math teachers with two years of professional development failed to discern a measurable impact on either teacher knowledge or student achievement.
The focus of the training was to better prepare teachers to cover rational number topics, such as fractions, decimals, percent, ratio, and proportion, which are seen as an important foundation for learning algebra but also a common stumbling block for students.
The point of the research exercise, conducted by the American Institutes for Research and MDRC, wasn’t simply to make a judgment about one professional development offering.
“The PD program in the study had all the features that accumulated research to date said are important to making PD effective, so it’s not just that we arbitrarily picked one,” explained Andrew Wayne, the deputy director of the research project and a principal research analyst at AIR. “The field would have expected a program with these features and this intensity to have a statistically significant impact.”
Alas, it did not (with one exception related to teacher knowledge—but still not student achievement—outside the main analysis).
“The biggest takeaway is that we don’t yet know how to use professional development at a large scale to reliably improve teacher knowledge and student achievement,” Wayne said.
The program studied was “far more intensive and extensive—and better—than the typical professional development” that teachers receive, noted Elizabeth Warner, an economist at the federal Institute of Education Sciences’ National Center for Educational Evaluation and Regional Assistance, and the project officer for the study. “It’s very much different from business as usual.”
The professional development was provided by two organizations, America’s Choice and Pearson Achievement Solutions, selected through a competitive process guided by a panel of outside experts.
The study focused on roughly 100 teachers who received the training in 12 school districts in the first year of the study, and about 50 teachers in six districts during the second year. They were compared to control groups of equal size with teachers who did not participate in the program.
The researchers cautioned that staff changes over time prevented some participants from getting the full benefit of the training over two years. More specifically, about half of the teachers who were in the second year of the program were not there for the first year.
However, the researchers suggested that this type of variation is pretty consistent with what would happen in a typical school district.
The study found no statistically significant impact on teacher knowledge, as measured by a specially constructed test, from participating in the professional development. Also, after the second year of implementation, the program did not have a statistically significant impact on average student achievement regarding rational numbers.
The program was designed to provide participating teachers during the two years with a combined total of 114 “contact hours,” including summer institutes, one-day follow-up seminars, and in-school coaching visits. However, not all teachers participated for the duration, largely because of staff changes from one year to the next.
The study used an experimental design with random assignment of schools to treatment and control conditions within each participating district.
The research project did reveal some limited results that were more promising, based upon an “exploratory” analysis that drew on a larger sample of teachers, including teachers who were in the first-year sample only, teachers who were in the second-year sample only, and those in both groups. Here, the estimated effects from one year of the professional development program did show a statistically significant impact on one dimension of teacher knowledge, what it calls “specialized knowledge.”
Basically, the researchers sought to distinguish between “common knowledge” of rational number concepts and “specialized knowledge.” The former is straightforward understanding of rational number content. The latter is what Wayne decribed to me as “rational number-related knowledge that helps you teach it."This includes, for example, knowing what types of graphic representations will best convey specific ideas clearly, and knowing the common student misunderstandings about this content.
The findings were not much different from those in a similar study commissioned by the IES on early reading, completed in 2008. That analysis showed no statistically significant impact on student achievement after teachers were exposed to one of two year-long staff development program in the subject. And while a positive impact was detected on “some dimensions” of teacher knowledge and instructional practice at the end of the year in which the training was delivered, the benefits were not sustained a year later.
Stepping back, the report notes that in general, there has been “only limited research evidence” to date about the impact of PD on teacher and student outcomes.
For still more about what the research says on professional development for teachers, check out my colleague Stephen Sawchuk’s story from last November. His conclusion? Much the same, that solid evidence appears to be lacking about the efficacy of such extra training for teachers.
One last thing from the federal study issued this week: The exploratory analyses did find evidence to support one notion that a lot of people surely believe to be true: Students taught by teachers with higher knowledge scores showed significantly higher achievement (after controlling for prior achievement and other factors).
“We have to get more bang for our buck in improving teacher knowledge, we need to get that [impact] to be bigger, and get that well translated into the classroom,” Warner said. “And I don’t think we know how that happens.”