Note: Morgan Polikoff, assistant professor at the University of Southern California, is guest posting this week. You can follow him on Twitter at @mpolikoff.
For my last day here at RHSU, I’m going to go a bit into the weeds to talk about what I think are the two most important technical issues that will affect Common Core implementation. These are the quality and alignment of both assessments and curriculum materials. This post is based on an upcoming brief I’m releasing through the Center for American Progress on assessment in the Common Core era, as well as a research paper on textbooks I am presenting today to the Education Writers Association here in Los Angeles.
As is well known by now, assessment in the NCLB era suffered from a number of serious problems. For example, state tests were poorly aligned with the state standards they were intended to assess (which undermined standards implementation). They routinely focused on content on lower-level skills such as memorization and procedures and left vast swaths of the standards unassessed. They tended to reliably sample the same content over and over again, leading to score inflation. The proficiency cuts were all over the place, so proficiency became basically meaningless. Furthermore, the requirement to assess only grade-level standards led to poor measurement at both the top and the bottom of the achievement distribution. Finally, many were dissatisfied with the almost exclusive emphasis on multiple choice items, which may be limited in their ability to measure certain complex skills.
The two assessment consortia are working hard to improve upon prior state tests, and there is good reason to think they will succeed at least somewhat in their efforts. For instance, they are building constructed response and performance items to assess hard-to-test standards. States participating in the consortia are expected to adopt common proficiency targets. And computer-adaptive testing (in Smarter Balanced) could help with assessing achievement at the extremes of the distribution. It goes without saying that the consortia need to create strong tests that cover the full domain of the standards, including the more conceptual skills. If the tests end up looking like NCLB bubble tests on a computer screen, they will certainly undermine the standards. I offer more specific assessment suggestions in my forthcoming CAP brief.
The second major threat to Common Core implementation lies with curriculum materials. We’ve all heard the anecdotes of publishers slapping “Common Core aligned” stickers on everything with words. There should be real concern about these claims, and I set out to begin evaluating them in a recent project.
While the research on curriculum materials is not as great as that on assessments, there is no question that textbooks have historically suffered from many of the same tendencies as standards and assessments--excessively broad and shallow, covering too many topics at a superficial level of depth. We also know that textbooks vary in their effects on student achievement, and scholars have recently called for more research and data on textbook quality and effectiveness.
As a proof of concept for what I hope will be a larger study, I chose four popular textbook series published by Pearson, Houghton Mifflin Harcourt, Macmillan/McGraw Hill, and Saxon at one grade level--fourth grade mathematics. The first three claimed Common Core alignment and the last did not. I worked with teams of content analysts to examine the books from cover to cover using the only established alignment technique shown to be useful for analyzing alignment among instruction, curriculum materials, standards, and assessments. We also analyzed the Common Core.
The results of our alignment analyses, which I’m debuting today at a meeting of the Education Writers Association (and presenting at a couple conferences this spring), are both stunning and totally predictable. Despite the presence of the “Common Core aligned” labels, all textbooks studied had notably large areas of misalignment. All textbooks systematically failed to include content (far less than 1% of total content) on advanced levels of cognitive demand like generalization and application, even though about 10% of the standards content is on those skills. A substantial chunk (10-20% of the textbooks’ 100+ lessons) was on content not in the corresponding standards, and about 15% of the standards were not in the texts. Finally, I compared the textbooks to prior versions of the same textbook series that claimed alignment to Florida’s standards. I found substantial agreement (60-95%, depending how defined) indicating that publishers used much of the same material to cover Common Core as to cover Florida’s standards. This is despite the fact that the two sets of standards ask for very different content.
I hope this work will be the first in a line of research investigating curriculum materials, their content, and their influence on student learning. Given the ubiquity of these materials and their effects on teachers’ instruction, there is a great deal more to be learned. As with assessments, having high-quality, aligned curriculum materials will almost certainly be essential to helping teachers understand and teach the standards. And if the standards aren’t well implemented, they will not have the effects many are hoping for.
--Morgan Polikoff