Recently in Assessment Category

When I wrote about the draft of the national assessment in technological literacy that is scheduled for 2012 it was summer break for most educators and few people in the field had had time to digest its contents. But now some education technology experts are expressing concerns that the first National Assessment in Educational Progress to gauge students' understanding of and proficiency at using technology will be inconsistent with common practice in most American classrooms.

You can catch up on the conflict with this post on the Curriculum Matters blog, by my colleague Sean Cavanagh. He outlines the sharp criticism coming out of the State Educational Technology Directors Association.

Posted by Kathleen Kennedy Manzo at 3:50 PM | Permalink | No Comments | TrackBacks (0)

Sean Cavanagh, my colleague over the Curriculum Matters blog, has this report from the National Assessment Governing Board meeting:

There's a lot of debate these days about how to define "technology literacy," but in a couple years, the National Assessment of Educational Progress will take the unusual step of testing students in those skills.

This week, the panel that oversees the National Assessment of Educational Progress heard an early report on how it is attempting to forge a working definition, in preparation for judging students' tech literacy in 2012.

The National Assessment Governing Board, which sets policy for the NAEP, must first develop a framework, or basic blueprint for that test. The board has put together steering and planning committees to work on the project. Those panels include lots of familiar names in education and school technology fields, including Don Knezak of the International Society for Technology in Education, Mary Ann Wolf of the State Educational Technology Directors Association,Senta Raizen of WestEd, who's co-chairing the committee, and many others.

On March 6, governing board member Alan Friedman, a science and museum consultant from New York who is working on the tech literacy test for NAGB, talked about how the board is going about that task. A prime challenge is developing a definition that will stand the test of time, Friedman said, so that the test is not outdated within a few years after it's been unveiled.

Despite the name of the test, Friedman made it clear that goal of the NAEP tech literacy exam is not simply to test students' familiarity with computer products or features, or digital games. The goal is to evaluate their understanding of "interconnections among technologies," with technologies including processes from the designed world, he said. This could include not only computers but technology's relationship to processes such as metallurgy (in the manufacture of buildings, or individual products) or woven textile technology (used to make clothes and fabrics). Of course, computer technology is essential to many manufacturing processes today, noted Friedman, who was joined by Raizen in his presentation. But the point is that students need to have a broader grasp of technology that takes them beyond their computer keyboard, if they're to understand complex scientific issue today.

"We need to understand what all technologies have in common, and how they inter-relate," Friedman told the board. It's likely to be a major task, he suggested. "This project is working with probably more of a blank slate than any other framework we've developed."

The governing board awarded a $1.86 million contact to WestEd last year to develop the framework and test specifications. The committees are expected to deliver a framework to the full governing board by November of this year.

Posted by Kathleen Kennedy Manzo at 9:02 AM | Permalink | 3 Comments | TrackBacks (0)

A couple weeks back, Education Sector released a new report about the role of technology in assessment. The report, "Beyond the Bubble: Technology and the Future of Student Assessment," talked about how technology could be used to automate assessments to provide quicker feedback. But it also looked at how it could help transform assessment to provide meaningful data on students' test answers and, perhaps more importantly, how they got them.

From today through Thursday, Education Sector is hosting a discussion about the report with its author, Bill Tucker, as well as education experts Charles Barone, Margaret Honey, and Scott Marion. Readers are encouraged to submit questions and join into the discussion.

Posted by Katie Ash at 12:13 PM | Permalink | No Comments | TrackBacks (0)

The Oregon Department of Education is not scoring very well in its 2-year-old dispute with Vantage Learning.

On Friday, an Oregon jury awarded $3.5 million to the online testing company to compensate it for tests that it had administered but that were not paid for by the state.

The jury of the Marion County Circuit Court found that the education department breached its contract with Vantage by failing to negotiate in good faith to settle the payment issue, establish a new pricing structure, and deal fairly with the company.

The jury unanimously rejected the education department’s claim that Vantage Learning breached the contract and dealt in bad faith.

The state in March 2007 sued Vantage, which is registered in Delaware but has its headquarters in Newtown, Pa. Vantage had been the state's partner since 2000 in developing and launching a new Web-based testing system, called Technology Enhanced Student Assessment. For several years, TESA was used to deliver state tests in reading and math in grades 3-8 and grade 10, as well as science in grades 5, 8, and 10.

The dispute arose over issues of system performance and payment. But a significant fact was that, when the state rebid its testing contract, it disqualified Vantage from competition in the fall of 2006. The state later chose the current test provider, the American Institutes of Research, based in Washington.

In the most dramatic incident, Oregon students who logged into the testing system on March 9, 2007 were presented an on-screen message from Vantage, explaining that the test was unavailable because the state had not paid its bills.

You can read my story for Education Week here.

A story this week on the jury verdict in The Oregonian is here.

In 2007, Oregon education officials took an aggressive stance against the company, with Deputy Superintendent Ed Dennis contacting the news media to describe the state's complaints. Vantage, perhaps to minimize negative publicity, kept fairly quiet.

This week, however, it was Vantage that was crowing about its courtroom victory.

President Robert Patrylak said to me, in an interview, that the jury was swayed by "the fact that [the state] never came to sit down with us, even after we gave opportunity after opportunity to have them do that."

Harry N. Barfoot, the company's vice president, said that Oregon's current online testing system is costing the state's taxpayers from three to five times the cost of TESA.

And Barfoot claimed the high ground in the dispute: "The lesson learned here is it does matter to taxpayers, to a company like Vantage, and on both sides, to have a program that works, to have fair contracts that are adhered to, with issues that are worked on, and written on a piece of paper that means something."

Oregon's education officials have not yet decided whether to appeal the verdict.

Dennis issued a short statement, expressing disappointment in the verdict, and reading in part: "... We filed suit because Oregon’s students and teachers suffered as a result of what we still believe to be a breach of contract by Vantage Learning. This breach resulted in districts scrambling to administer paper and pencil assessment tests during the spring of 2007. Fortunately, we have a new contractor delivering our online assessments and have not had any further disruptions for our students. We will meet with our legal counsel from the Department of Justice to explore our options for appealing this decision and continue to press our case.”

Posted by Andrew Trotter at 3:37 PM | Permalink | No Comments | TrackBacks (0)

Vint Cerf, who is often called the "father of the Internet" for his contribution to creating its technical protocols and architecture, will have a hand in developing a framework for the first nationwide technology literacy assessment of U.S. students, as part of the National Assessment of Educational Progress.

That's the inside scoop from Steven A. Schneider, of WestEd, which on Oct. 6 was awarded the contract by the National Assessment Governing Board to develop the framework and specifications for the test.

The assessment, which will be first offered on a pilot basis in 2012, will be "totally computer-based," Schneider, WestEd's senior program director for math, science, and technology, told Digital Education.

WestEd, located in Redwood City, Calif., developed the framework for the NAEP science assessment.

The NAGB has stated that a technological literacy assessment is needed to understand U.S. students' ability to compete in a global marketplace and keep pace with quickly evolving technology.

Cerf, 65, a recipient of the Presidential Medal of Freedom who now works for Google Inc., will be on the steering committee for the 18-month project to develop the framework. There will also be a planning committee that meets more frequently.

Names of other members of the committee have not been released but will represent a broad spectrum of expertise, Schneider said. Both panels will have their first meeting in mid-December.

Schneider added that the effort will draw on many experts and groups, including the International Technology Education Association, the International Society for Technology in Education, the Partnership for 21st Century Skills, the State Educational Technology Directors Association, and the Council of Chief State School Officers, as well as the more than 10 states that now have assessments for technological literacy, he said.

Posted by Andrew Trotter at 3:39 PM | Permalink | No Comments | TrackBacks (0)