Statistics Czar Critiques International Exams
Mark Schneider, the former statistics czar for the U.S. Department of Education, offers an interesting critique in Education Next of efforts afoot to enlist states to participate on their own in international assessments, such as TIMSS and PISA. (For the uninitiated, TIMSS stands for the Trends in International Mathematics and Science Study and PISA is the Program for International Student Assessment.) Schneider writes:
What would be gained if, in addition to the nation as a whole, individual states were to participate directly in these assessments by testing a much larger and more representative sample of students? Not as much as many advocates would have us believe, and probably not enough to justify the considerable cost.
He offers two reasons for his skepticism. First, it's likely that most states won't have much to crow about in the results. Studies that use existing data from the National Assessment of Educational Progress to link student achievement in states to that of other countries have shown that even the best states can't match high-performing Chinese Taipei or Korea.
Second, he writes, the results can't offer much in the way of sound policy guidance for educators because of their statistical limitations. On this point, Schneider is particularly critical of PISA, which he says routinely issues sweeping policy conclusions without offering any qualifications.
The cost to a state for fully participating in TIMSS or PISA can be as much as $500,00 to $700,00 per grade, according to Schneider. A less expensive way to go, he added, might be to use statistical measures to link data from NAEP to international-exam results for other countries. If states still want to pony up to expand their participation in international assessments programs, though, Schneider's advice is caveat emptor.