Standards

D.C. Will Invalidate 2 Years of Science-Test Scores

By Stephen Sawchuk — January 19, 2018 3 min read
  • Save to favorites
  • Print

UPDATED

The District of Columbia will cancel two years of students’ science assessment scores, alleging that its test contractor, WestEd, bungled its handling of the tests.

“Serious errors” in the tests will mean invalidating the 2016 and 2017 scores; the 2017 scores will not be publicly released, officials from D.C. Office of the State Superintendent of Education said in a Friday news release. What’s more, the D.C. school system won’t be administering science assessments this year, and will seek out a new test creator moving forward.

The news comes just as many states have begun to overhaul their science standards and put in place matching tests. The District of Columbia was one of the first places seeking to measure students’ grasp of the Next Generation Science Standards, which have been adopted in about 18 states.

In Washington, science tests are administered in grades 5, 8, and the year the student takes biology at the high school level. About 13,000 students take them annually. Federal law requires all states to administer science assessments three times over a student’s trajectory—once in the elementary grades, once in middle school, and once in high school.

The agency has cancelled its contact with WestEd, worth $370,000 this year, which covered the construction and scoring of the exam. It is also hinting at a possible lawsuit: OSSE is working with D.C.'s procurement office to seek legal remedy, including possible damages.

“Assessments are an important part of understanding how well our students are doing and we, our educators, and our students devote significant time and resources to taking them seriously,” D.C. Superintendent of Education Hanseul Kang said in a statement. “Any action by a vendor that contradicts that commitment is unacceptable.”

Federal Waiver?

The delay caused by the errors means that D.C. will need to seek approval from the U.S. Department of Education to forgo science testing for 2018. In the release, OSSE said it has worked “amiably” with the Education Department to meet its reporting requirements under the law, known as the Every Student Succeeds Act.

The cancellation doesn’t otherwise affect D.C.'s plan under ESSA; unlike other states, the district decided not to factor science-test results into school ratings.

In an FAQ also posted on the OSSE website, OSSE said it noticed during an audit “unusual results” on the 8th grade test compared to the prior year and to other grade levels, and that further inquiry by OSSE and WestEd found additional problems. In an email, the agency said that the errors involved scoring and “psychometric services"—which generally means making sure that the test meets validity and reliability standards and is fair for different groups of students—not something intrinsic to the test items.

The Washington Post reported that the errors made it impossible to compare the 2017 and 2016 results. That appears to be what’s called in testing jargon a problem with equating, or ensuring that scores are comparable from different test forms.

Max McConkey, the chief policy and communications officer for WestEd, said that the organization didn’t agree with all of OSSE’s conclusions. “We believe we’ve responded to concerns in the past, and have a very different interpretation of the history,” he said. “We just couldn’t resolve it in a way they were happy with.”

OSSE said in its statement it had already put out an RFP seeking a new vendor for a replacement exam, to debut in 2019.

The NGSS standards put an emphasis on teaching students the scientific process—including how to craft a hypothesis, conduct experiments, and analyze results—as well as on scientific content. Only a handful of other states were on track to use an NGSS-aligned test in 2018; one of them, Illinois, had an agreement to use some of D.C.'s science-test questions, but OSSE officials said that that state uses a separate contractor and test design, so it shouldn’t be impacted by these errors.

As we’ve reported, the standards necessitated testing experts to think about how to create tests that could measure across the NGSS’ science practices, content, and “cross-cutting themes” like engineering.

This story will be updated.

A version of this news article first appeared in the Curriculum Matters blog.