Reading Tests: Know What You're Measuring
Back in July, I wrote a blog post about a change in accommodations for students in Delaware taking a high-stakes reading test. The state allowed fewer students with disabilities to have the reading test read aloud to them, and officials were attributing a drop in test scores to that change.
I know that I was left wondering how reading aloud a reading test could have ever been a valid test accommodation. Hearing the words ultimately changes the test in a fundamental way, doesn't it?
Now I know the answer, which is: it depends.
Jennifer Randall, a faculty member at the University of Massachusetts Amherst's Center for Educational Assessment, was the lead author of a study just published in August 2010 issue of The Journal of Special Education. In talking with her, I learned much more about test accommodations as well as the complexity of testing.
In her study, Randall tried to find out if read-aloud tests provided a "differential boost" for students with disabilities in Georgia. About 2,000 students were a part of the study.
First, all the students took the test, without any accommodations at all. Then, the test was administered again, with the accommodation.
Valid accommodations should boost the scores of students with disabilities (the "differential boost") while leaving the scores of general education students about the same. From that result, researchers can tell that a particular accommodation is helping the students that it is meant to help.
Randall's study measured two separate accommodations: reading aloud a reading test for 4th graders and 7th graders; and providing a "resource guide" that 4th and 7th graders could consult during their test that was supposed to provide students with information they could use to answer the questions, but not the answers themselves.
She found that there was a differential boost among 4th graders with disabilities when the test was read aloud. That indicates that reading a test out loud for those younger students may be a valid test modification for them. But for 7th graders, the scores of students with and without disabilities went up when the tests were read aloud. That means read-aloud could be an unfair advantage to older students with disabilities who get read-aloud accommodations when their non-disabled peers do not.
As for the reading guide, special education teachers had been clamoring to offer that particular accommodation, Randall said. But in her study, scores went down among students with disabilities who used it. She suspects that the students had little experience with the guide and it served as a distraction. And for students with reading problems, the guide was just one more item to read.
As for reading aloud, what's important is what the test is intended to measure, Randall said. If the test is meant to assess a student's ability to decode words, then reading a test aloud wouldn't make much sense. But if the intent is to see if a student, when presented with information, can draw correct conclusions and comprehend certain facts, then read-aloud might work.
"As large a deal as large-scale state assessments are, before [states] take away accommodations or add one, there should be an experimental design. It needs to be about what your test is," Randall said.