Beyond Test Scores: Rethinking Accountability
This week, officials from about a dozen states are gathering in Milwaukee for the annual meeting of the Innovation Lab Network (ILN). The ILN is a group of states, formed by the Council of Chief State School Officers (CCSSO), that are developing new approaches to policy to support innovative methods of schooling intended to support deeper learning outcomes. (The ILN and its director, Jennifer Davis Poon of CCSSO, are regular contributors to this blog. See here and here, for example.)
One issue on the agenda is accountability. Educators have long recognized that current accountability systems, which generally measure school performance solely on the basis of end-of-year reading and mathematics tests, are impediments to innovation because they create incentives to focus on a relatively narrow set of knowledge and skills. Moreover, such systems provide little information to enable school communities to examine practices and improve their own capacity to raise the level of student learning.
A year ago, a paper by Linda Darling-Hammond, Gene Wilhoit, and Linda Pittenger outlined a vision for a new accountability system, one that is oriented toward higher learning outcomes and that is designed to enable continuous improvement. Using that paper as a model, a group of states, many of whom are part of the ILN, have been working to revise their accountability systems, with support from CCSSO; the Stanford Center for Opportunity Policy in Education (SCOPE); the Stanford Center on Assessment, Learning, and Equity (SCALE); the Center for Innovation in Education at the University of Kentucky; and my organization, the Alliance for Excellent Education.
This group of states are considering several changes to their accountability systems. One is to incorporate multiple measures of school performance and to display them on a "dashboard" that provides transparent information for schools and communities. In a paper released earlier this year, the Alliance suggested that by incorporating multiple measures, such dashboards can address some of the shortcomings of current systems, such as the lack of transparency, the lack of guidance for improvement, and the inability of local districts to identify their own measures of performance. The paper also outlined some of the issues states need to consider in developing multiple-measure dashboard systems:
- Choosing the right indicators. Dashboard systems should include measures that matter for school improvement, but states need to be judicious to avoid overloading dashboards with too many indicators that make them difficult to use.
- Identifying the most critical indicators for special scrutiny. States, districts, and schools focus on specific key indicators to ensure that schools pay particular attention to them.
- Setting appropriate targets or benchmarks. Goals for improvement should be ambitious but reasonable.
- Using the dashboards for improvement. Administrators and schools work together to develop improvement plans rather than point fingers and assess blame.
Using waivers from No Child Left Behind, several states, such as Kentucky and New Hampshire, and large districts, such as the districts that make up the California Office to Reform Education (CORE), have been developing multiple-measure dashboards to replace NCLB-style accountability systems. These efforts have raised additional issues. Two of the most prominent issues concern the following:
- Hard-to-measure indicators. The CORE districts' dashboard, for example, includes measures of social-emotional learning. Many educators agree such measures are important, but there is not yet agreement on the best way to measure these competencies.
- Balancing state and local measures. New Hampshire's pilot program in four districts includes a mix of locally scored performance assessments with statewide assessments. The state measures are designed to ensure equity across districts, but the locally scored measures allow the use of performance assessments that are difficult to administer statewide. How to incorporate the latter while maintaining the former can be challenging.
Over the past year, the ILN states and their partner organizations have explored these and other issues associated with multiple measures dashboards through webinars and convenings. In the coming months, the states and their partners will work in greater depth to examine the states' efforts to put these systems in practice. Their work will set a path that other states might want to travel in the coming years.