If you were a big government agency with $350 million to invest in shaping a new generation of student assessments, what would be the smartest way to proceed? That was the main question floating around a hearing room in Washington today. And it generated far more questions than answers.
Today marked the start of the second round of hearings the U.S. Department of Education is hosting to get input from experts and the public to help shape the competition for a special slice of Race to the Top money that's earmarked for development of common assessments aligned to common academic standards. (See our story from the first round here, and my blog post about the announcement of this round of hearings here.)
Today's session sought advice on how the consortia should be organized and what they should have to demonstrate to be viable competitors for the money.
Five experts in consortium management shared their thoughts with three Education Department officials: RTTT Director Joanne Weiss, Deputy Secretary Tony Miller, and Deputy Assistant Secretary Judy Wurtzel. They agreed on some things, like the importance of states' having a clear, shared vision of the testing system their consortium aimed to produce, and clearly defined roles of who would do what to get the job done. But thorny, unanswered questions outnumbered those agreements.
Like this one, from Weiss: How should we shape the application so that we can tell whether states are truly committed to this work? Or this one, from Miller: In deciding which consortia deserve grants, how do we assess states' leadership and technical capacity to design and build good testing systems?
On Weiss' question, Michael Cohen, the president of Achieve, which leads a 5-year-old consortium that developed shared math assessments for 15 states (and is co-leading a separate effort to draft common academic standards), urged federal officials to ask applicants for a clear description of the testing system they plan, and how the consortium will be governed. They should also ask how other aspects of the education system in their states (things like curriculum, instruction, and professional development) must be changed so that the new tests actually improve student achievement.
Marc S. Tucker, the president of the National Center on Education and the Economy, which oversaw a standards-and-assessment writing consortia in the 1990s, told Weiss that the only way to gauge states' commitment and capacity is to send teams out to talk to key officials, to see if they're really passionate, informed, and on the same page. "Whatever papers you ask people to sign in the process are worth almost nothing," he said.
To Miller's question, Cohen posed a conundrum: Have states describe their technical capacity in detail, he said, and then assume they're underestimating it. The challenge for the department, Cohen said, is pressing applicants to be specific enough without making their applications into "works of fiction" because there are some things they just won't know this early on, and allowing them enough wiggle room that they can provide their best guess of what they can do, without having their applications become "boilerplate." He confessed that he didn't have a simple answer for that dilemma.
There was some interesting feedback, also, from experts from two management-consulting groups, McKinsey & Co. and the Parthenon Group, on the characteristics of successful and unsuccessful consortia, and the leader of WIDA, a 22-state consortium that developed standards and assessments for English-language learners. Tim Boals, WIDA's executive director, advised the Ed Department leaders not to presume that states can manage the tests once they've been developed, and warned them against letting each state in the consortium customize the common tests too much.
Tomorrow's meeting focuses on procurement issues that could crop up when groups of states work together to develop tests, as they are required to do to get a chunk of the RTTT money. A Jan. 20 session will probe general and technical questions.
The list of experts for these hearings is available on the department's RTTT assessment page. Their Power Point presentations will soon be available as well, and the sessions will also be available for viewing as a webinar.
The department is also accepting written input until Jan. 20. Curiously, the department received so few requests from the public to give input at today's meeting that seven of the 12 available five-minute slots for speakers went unfilled, and the meeting adjourned early. And the agency doesn't plan to do a public-comment period once it drafts the regulations.