What makes for robust school accountability systems from states, and where do parents and researchers differ about which states do the best job with those reports? The Education Commission of the States has issued a report trying to answer those questions, and also raise new issues for policymakers to consider as states (in ECS’ view) enter the era of “Accountability 5.0.”
I touched on a similar report ECS put out for 2013 that detailed the extent to which states are using common metrics in their accountability systems—last year, for example, all 50 states reported using test scores or some form of student performance in accountability. In fact, both the 2013 report and the group’s most recent look at accountability, show a broadening consensus in terms of what’s being measured by states for accountability, as well as what’s being reported (but not necessarily being used in accountability formulas). Here’s the most up-to-date look ECS has put out:
The purple column highlights the five indicators ECS considers crucial to building a strong accountability system. Which states collect, use, and report them all? According to ECS, California, Colorado, Kentucky, New Mexico, North Carolina, Ohio, Oklahoma, Pennsylvania, Utah, and Wisconsin do (Ohio’s new accountability system will be fully online in 2015, so ECS included it).
But those discrete metrics aren’t the only factors when one is considering “strong” accountability. There’s also the question of whether they’re easy to read and understand, and subsequently, whether the public truly believes in the results on display. “Letter grades are easiest for parents and other constituents to understand. But if a clear rating sits atop a hill of measures that communities don’t trust, questions are likely to follow,” states the report, which was written by Marga Mikulecky and Kathy Christie at ECS.
What are some of the common complaints? ECS lists them:
• The metrics aren’t right. For example, too much emphasis is placed on test performance and/or
too few subjects are tested.
• The metrics, weights, measures, and formula do not accurately reflect school performance.
• Composite scores are seen as less transparent and nuanced than separate indicators.
• Communication about how the grades are determined is vague or inconsistent.
• Even a rocket scientist can’t figure out the formula.
• The metrics, weights, formula, and report card do not reflect public values.
With those issues in mind, ECS separately asked a group of researchers and a group of parents to review state accountability reports and decide which ones were the easiest to understand and were most useful.
For researchers, the three states with the strongest accountability reports on those criteria were Arizona, Illinois, and Ohio, with Delaware, Kentucky, Louisiana, Massachusetts, and Maine also getting honorable mention. For parents, Delaware, the District of Columbia, and Illinois were the top three, with Arkansas, Ohio, and Wisconsin also receiving recognition. So Delaware, Illinois, and Ohio seem like good resources for providing broadly useful accountability reports, according to ECS, with Illinois standing out as the “clear winner” for making both groups’ top-three lists.
One parent’s comment about the Illinois system was that it “provided directions as to how to navigate the page and was not overwhelming with data,” while one researcher remarked enthusiastically, “THE BEST SO FAR. Easy to interpret, everything is clickable for more information.”
In their conclusion, Mikulecky and Christie note that while there’s no such thing as a perfect K-12 accountability system, some states may need to reconsider the transparency and usefulness of their systems, and to continue wrestling with the question of “to what end are schools being graded?” That’s a question that will likely continue to bedevil states, and it remains to be seen if the consensus that has developed concerning which metrics to use for accountability will also develop for the consequences determiend by accountability systems.
For the complete report, see below (the Illinois accountability system is reviewed on page 11 of the report):