California Trying to Build Decentralized Evaluation System
What if the problem wasn't herding kittens, but evaluating the play routines of each of them?
In a way, that's the problem California faces. It has radically decentralized fiscal authority and accountability with the Local Control Funding Formula and local control accountability. It has ditched its single academic indicator that was used to pass out the "failing schools" badges along with sanctions. In its place, California has adopted Eight State Priorities, and even these are somewhat subject to locally devised measurements.
Two of the state's leading policy thinkers have taken a stab at how the state can conduct evaluation in this context and create a virtuous circle of continuous improvement. In a paper released today, Linda Darling-Hammond and David Plank—both at Stanford University—describe the key elements in the state's work-in-progress accountability system.
As the graphic above shows, their cycle of improvement is familiar and sensible to organization designers and systems thinkers. Learning supports, information systems, ongoing reviews, innovation and evaluation, and knowledge sharing all support one another.
The problem is that none of those sub-systems exists yet, and meanwhile the young cats are beginning to play. So, there is some urgency to the issue.
Darling-Hammond and Plank argue that the state needs "two key pillars to support continuous improvement":
The first is an information system to replace earlier competing systems and measures with a dashboard.
"The state should replace the Academic Performance Index (API), the State Accountability Report Card (SARC), and the current on-line reporting system with a dashboard of measures that reports progress on the state's priorities.
Instead of seeking to rank schools and districts on a single measure, the dashboard will reveal how they are doing in relation to criteria for performance and how they are improving in different areas. The use of multiple measures is much more informative than a single index for planning and improvement efforts. Like the dashboard on a car—which provides indicators of speed, distance traveled, fuel, fluids, tire pressure, and more—the combination of measures provides information about where to look further in order to figure out how things are working and what may need attention."
The second is a new state agency, which already exists in law but not in practice. The California Collaborative for Educational Excellence (CCEE) is supposed to create supports for learning, knowledge sharing and evaluation, as well as direct intervention.
Under the local control financing statutes, CCEE is to provide "direct assistance" to those "falling short of their goals and obligations." The authors lay out a School Quality Review process for undertaking this task using peer reviews, distinguished practitioners and lots of data.
They note that this approach is already in use in California by the districts involved in the California Collaborative for Educational Excellence, and similar efforts were pioneered in Shanghai, China.
Will this approach simultaneously herd and evaluate the kittens? (I encourage readers to study the policy brief and the full report and think through the problem for themselves.)
∞ ∞ ∞ ∞
My initial take: this system depends on resources that the state does not yet have, and will require lots of unlearning on the part of the California Department of Education.
When it comes to water and trains, Gov. Jerry Brown understands infrastructure. I'm not sure that he understands that designing and running a decentralized education system requires one, too. There needs to be a serious discussion about how one funds CCEE and the evaluation system.
If this discussion does not take place and a robust evaluation and assistance system is not developed quickly, the scores on the new Smarter Balanced tests will be the state's de facto evaluation system, and it will have simply created a revised version of the No Child Left Behind era, something neither the governor nor the education establishment does not want.
The second task—unlearning—is harder. Chills ran down my back when I looked at the School Quality Review graphic (above). I've seen this movie before, and so has every educator in the state. From program quality reviews to accreditation, schools and districts have been subjected to scores of external examinations for decades. They become routinized, and efforts to make them coherent end with universal prescriptions.
If the new reviews are going to be carried out by the people who carried out the old ones, they will likely look like the old ones. It's less people's organizational structures that need changing than their mental ones.
∞ ∞ ∞ ∞
Finally, the continuous improvement system largely omits the actual learning system. In trying to redesign its education system, California has built two legs of a three-legged stool. It adopted high standards and better standards than it had before that will encourage deeper, more performance based learning. And, as the Darling-Hammond and Plank policy papers illustrate, there is active thinking about how to create a new evaluation system. But relatively little of this directly impacts changing instruction and how it is organized in schools.
In a paragraph well into their paper (p. 27) Darling-Hammond and Plank say:
[T]he CCEE could help to rebuild some of the now-missing infrastructure for professional learning in California. Over the last decade, the state has lost or greatly reduced most of its programs supporting professional learning for teachers and administrators, including the California School Leadership Academy, which trained leaders as well as teams for school turnaround, the California Subject Matter Projects, and the many professional development programs previously attached to categorical funding streams that are now included in the LCFF.
The stool won't stand until we rebuild the instructional leg. And the cats are unlikely to learn any new tricks.