College-Readiness Assessments: Be Careful What You Wish For
This post is by David T. Conley, the president of EdImagine and a professor at the University of Oregon, and Lydia Dobyns, the president and CEO of NewTech Network.
The arguments nationally about assessment are tending toward overly simplified characterizations: we're testing too much, all high stakes testing is bad, just let teachers teach, all national assessment is useless and all local assessment is valuable. This pendulum swing away from the excesses of the past 15 years' assessment practices and toward a more uncertain future for assessment has yet to acknowledge the new challenges and opportunities that will accompany such a shift.
The experience of a group of schools that has a long history developing and implementing performance-based assessments provides some insight into the issues associated with new measures. The New Tech Network (NTN), a national non-profit school development organization, partners with districts that choose to implement its school model. That model consists of changes in three elements necessary to increase the proportion of students ready for college and careers: culture shifts in instruction, implementation of Project Based Learning (PBL), and new organizational structures. More than 175 elementary, middle, and high schools, primarily public schools, are members of the network.
Three years ago, NTN launched a small pilot of curriculum-embedded performance-based assessments with 18 teachers at 16 network high schools. These teachers used college readiness assessment (CRAs) in their courses. These CRAs had been developed by teachers experienced in PBL principles of effective task design gleaned from Literacy Design Collaborative materials. Pilot teachers graded these subject-specific tasks using rubrics co-designed by NTN, Stanford's SCALE, and Envision Schools.
The pilots demonstrated that performance assessments focused on college readiness and built around PBL can lead to rapid improvement in teacher understanding of student critical thinking abilities. Angelene Warnock, a chemistry teacher at Napa New Technology High School, for example, saw dramatic improvement in her students' critical thinking skills because, she said, for the first time, they came to know what critical thinking looked like and what it required of them.
Following the success of the initial pilot, all schools that subsequently joined NTN did so as "CRA Schools." Many existing network schools also became members of the CRA implementation cohort, which enabled their teachers to receive additional coaching support and training focused on CRAs. The cohort trainings led to a rapid increase in CRA schools. As of the 2015-16 school year, more than half of the 175 New Tech Network schools are implementing CRAs. This training has enabled teachers to develop high quality assessments, administer them in the context of their curriculum, and score them reliably. The result is that teachers now have far more information on the readiness of their students for college-level work.
It has been exciting to see how quickly teachers have understood the value of CRAs. However, the process has also resulted in unintended consequences and new challenges. Here are some of the most important ones:
Changes to Instruction: Curriculum and instruction have been most directly affected. Teachers are quick to realize that if they want students to demonstrate college readiness skills they have to provide instruction focused on that goal. Educators also begin to recognize that college and career readiness requires more than subject-area content knowledge. Students need to know how to collaborate efficiently, to speak and listen effectively, and to take ownership of and control over their own learning. However, simply agreeing that these outcomes are important isn't enough. Teachers need role models and opportunities for peer collaboration to design curriculum, develop new instructional techniques, and manage the performance-based assessment process.
Revised Teacher Learning Structures: Finding time to support adult learning is already a challenge in most PBL schools where teachers frequently create their own curriculum. When the demands of mastering performance assessments are added to the mix, time becomes even more valuable. Teachers need time to master and deepen their understanding of the college readiness rubric indicators. They need time to get feedback on their task designs. They need time and help to analyze the data they're gathering. And, finally, they need time to design new strategies that respond to what they learn from the data. The NTN schools employ a range of strategies to provide necessary time to teachers.
One such strategy is to allot time to look at--and learn from--student work. CRAs are rich in information about what students know, how they think, what they understand, and which skills they need to develop further. This is much more information than is generated by a standardized test. Once teachers have this information, they then need opportunities to act upon it. Creating the expectation that the results of CRAs will lead to changes in instruction is crucial to maximizing their effectiveness and legitimizing them in the eyes of teachers.
District and Leadership Alignment: Most NTN schools exist within larger systems, such as districts or charter management organizations (CMOs). The system must also adjust to support CRAs. These more authentic forms of assessment can be in conflict or tension with requirements that schools conform to centrally-managed benchmarking practices, curriculum pacing guides, or professional development schedules. Grading policies are another area of contention. Teachers often permit students to revise and resubmit their work, and it is more difficult to say when a final grade should be recorded, and whether the "grades" for each version should be noted on the record. An assessment system that uses CRAs opens the door to student portfolios and to competency-based grades as ways to measure student growth on complex skills that can only develop over time and need to be tracked longitudinally, not just checked off a list based on a single test score.
As district and school leaders agree on the outcomes they seek for students, the need for educators to focus educational practices on those ends takes on greater urgency. The Napa Valley Unified School District learned two years ago that when it focused on the larger goal of college readiness for all students and began de-emphasizing curriculum mandates, school teams were able to choose the approaches they deemed best to achieve the district-supported college- and career-readiness goal.
The act of focusing a school or district on the overarching goal of college and career readiness and adopting assessments that capture a more complete picture of student capabilities on key skills triggers a cascade of related changes in the classroom, the school, and the district. These changes can seem overwhelming. What the New Tech Network experiment with CRAs demonstrates is that teachers and school leaders can be highly effective developing and assessing the multidimensional skills needed for college and work life success.
Whether classroom embedded performance assessment is the remedy to the ills of standardized testing remains to be seen. In all likelihood, schools and policymakers will need to use information from multiple sources, including standardized tests, for the foreseeable future. The lessons that NTN schools have learned offer some insights into what it takes for complex assessment to work. We should be careful what we wish for if we think these assessments are a panacea that will solve all our testing problems. But if we use these complex measures as an additional source of insight into college readiness and as a way to reshape instruction, they may be able to support transformation in teaching and learning and improve postsecondary success for more students.