Accountability Systems Need to Help Schools Improve
California did away with its single number indicator system for school ranking, and there is widespread support for "dashboards" of multiple indicators to replace a single number to indicate how well a school is doing. So far, so good! But a bunch of numbers isn't enough.
Consider the dashboard on your car. Some of the indicators provide clear guidance about what to do. Drivers know to fill up the tank when the gas gauge gets low. They know that there are consequences for not paying attention to the indicator. Then, there are other indicators, such as the red overheating indicator light or the one for oil pressure, that tell the driver to call for help: stop the car, grab your phone, call someone who can help, maybe with flashing lights and a tow truck.
Better Numbers and More Help
Education accountability systems need both functions. Accountability systems are better when they provide numbers that people can use, but they also need to tell schools and districts when to call for help.
The advocates of the bill (AB 2548, Weber) going through the California legislature call it a cohesive system, but it's not really a complete system. The bill is headed to the Assembly Appropriations Committee on Wednesday. (Also, the State Board of Education will address accountability issues at its May 11-12 meeting. See Board President Michael Kirst on state commitments to closing the achievement gap.)
AB2548 would establish key indicators that meet the requirements of the new federal Every Student Successful Act and align federal and state accountability standards so that California schools will no longer face two standards. But, for the most part, it does not provide indicators that help a school improve or provide it the means. Without these, there's no coherent system, only wishful thinking.
Actually Useful Numbers
Let's talk about useful numbers first.
Key indicators are only the endpoint in a causal chain: the pointy end of the arrow that is on the public report card. Unless schools have a well-developed set of measures for those things that help the leading indicator, they will try to manipulate the indicator itself. We know from the NCLB era, that the result is likely to be cheating or lowering of standards. Counting the percentage of students enrolled in college prep classes (the A through G curriculum prized by the University of California) creates an incentive to re-label math and English classes so that they appear in the college prep category.
Creating systems of indicators that tell schools how to get better is an important task and one, as far as I can tell, overlooked by the Weber bill. The bill should be amended to provide incentives and capacity to develop these systems. (See our earlier post about adding narratives to numbers.)
Some Good Examples
There are good examples, some highlighted in a new publication from the Learning Policy Institute and SCOPE. Among those are systems in Alberta, Canada, in the CORE districts in California, and in New York. We've written about the CORE indicators previously, and in recent correspondence Noah Bookman, its chief accountability officer, provided an illustration of how test-measured achievement could be combined with other indicators to help schools learn.
In the example below, all the schools in the tall red box are at the bottom of the achievement scale, and under a cut-point accountability system would be treated the same even though some of them have done very well in other measures, avoiding chronic absences, for example.
But if one were to look at the same schools and ask which schools are doing a poor job on both sets of indicators, then those schools in the square box would be picked for an intensive intervention. The scatterplot presentation of results on two dimensions also directs our attention to the schools clustered at the top right hand part of the graph. It invites us to ask if these schools have programmatic similarities. Have they all invested in high quality professional development, for example?
Another example: The charter management organization, Summit Public Schools, is adding what it calls Habits of Success to its personalized learning model. It has already developed cognitive learning assessments. Summit is among the 12 grantees of the Assessment for Learning Project, whose goal it is to better connect assessment and personalized learning. (See Summit's rubric for social/emotional learning, along with those of other schools.)
There's a good bit of innovation being undertaken at the school and district level, and my hope is that the legislature will recognize the importance of these developments rather than retreating to test and punish.
Red Lights on The Dashboard
Now, let's consider what happens when the dashboard shouts "get help!"
California, like many states, has been through multiple iterations of school intervention systems, all with names provided by the bureau of acronyms, like SATE and DATE. In some cases, over $10-million has been provided paying outsiders to turn around a school. For the most part it's been a failure.
Now, the state is in the beginning stages of trying something new. The California Commission for Educational Excellence could have been another intervention agency, but its executive director Carl Cohn, the former superintendent of Long Beach and San Diego and former state board member, is determined to create an organization that helps schools and districts understand themselves well enough to get better.
Superman Not In The House
As Cohn told his board at its April meeting, "We don't see ourselves as superheroes parachuting in by ourselves. We are dead set on collaborating with those who can help us identify talent in the counties as we move forward."
Part of this is methodology, putting together the techniques that allow helpful outsiders to understand and guide a school or district to success using its own efforts. Fortunately, we know more about this process than we did a generation ago. Cohn told his board that he and his staff were reading the new Michael Fullan and Joanne Quinn book, Coherence, along with the board and staff of the Palo Verde School District in Blyth, which Cohn and colleagues visited last month.
Improvement Science, which is the basis of Coherence, has made big strides in the last decade, both as an academic field and as practice. It is radically different, and much more sophisticated than the processes involved in school interventions in the last several decades. Anthony Bryk, the director of the Carnegie Foundation for the Advancement of Teaching, is co-author of Learning to Improve: How America's Schools Can Get Better at Getting Better. It distills the lessons of four decades of hard, in-the-trenches work at improving schools.
CCEE's ability depends partly on good ideas, but it's capacity to help schools directly depend partly on the willingness of the legislature to steer necessary operating funds in its direction.
Unless there's help when the dashboard glows red, the accountability numbers become a cruel hoax.