Earlier this week, when the Smarter Balanced Assessment Consortium announced its initial cut scores, we got a glimpse of how student performance will play out on its tests in the spring. But the exercise of setting cut scores has done more than that; it has injected new vigor into a debate about what test scores mean and how they should be reported.
The quickest way to see the essence of this debate is to read a letter from Vermont Secretary of Education Rebecca Holcombe (see the letter embedded, below, or click here). Shared with the education chiefs from the other Smarter Balanced states, the letter lays out Vermont’s objections to the cut scores that the consortium was considering.
Holcombe didn’t oppose them because they were too high or too low; she objected because she views reporting scores by performance levels, with a threshold score for each level, as fundamentally unfair, inaccurate, and counterproductive for schools and students. She argues that reporting test results by scale score instead would yield more insight into student and school performance, and provide a better basis for constructive action.
“Reporting in performance categories has a public appeal: We like the apparent certainty and clarity of being able to categorize our students as ‘proficient,’” Holcombe wrote. “However, this misrepresents the underlying complexity of achievement and contributes to simplistic policies that make it difficult to achieve our public purposes.”
No Child Left Behind requires Vermont (and the handful of other non-waiver states) to report student achievement by performance categories, Holcombe wrote, so Vermont will comply with that requirement. But “the focus” of test-score reporting in that state will be scale scores, and particularly “mean scores and changes in individual and mean scores.”
“We believe this will support more responsible use of test data to inform improvement efforts at the school, district, and supervisory union level. We encourage our SBAC partners to also report using scale scores at the state level, so that we can collectively emphasize the commitment we share with respect to constructive use of assessments to support systematic improvements in learning.”
As we reported in our story, Vermont was one of two states that abstained when the schools chiefs in Smarter Balanced states voted on the cut scores. This is why. New Hampshire was the other state, and it abstained largely because its leadership felt it would have been better to set cut scores on the operational test, rather than on field-test data, Deputy Commissioner Paul Leather told me. But New Hampshire, too, is evaluating how it will report test scores. Leather said the state is considering reporting scale scores along with achievement-level categories, but the matter hasn’t been settled yet.
California is the biggest state in the Smarter Balanced consortium, and still bound by the reporting provisions of No Child Left Behind, since it doesn’t have a waiver. But education leaders there are pushing the boundary of that requirement, too; they led the way in creating a position paper, released with the Smarter Balanced cut scores, that urged states to take a broader view of what the test-score categories mean.