We had rather an interesting plenary at the CCSSO conference on student testing yesterday on international comparisons, and what the United States can learn from other countries’ education systems using exams like the Program for International Student Assessment, or PISA.
Here’s one way of slicing the PISA data that to me seems much more illuminating than the “rankings” of countries that seem to pop up everywhere in education debates these days: The PISA data can be broken down to show where a particular country’s strengths in a given assessment area are.
So, for instance, French students are good at identifying science issues and using scientific evidence, but their content knowledge of earth and space science and physical systems isn’t as strong. The Czech Republic is the exact opposite: Its students possess pretty good knowledge about science content, but they’re not as good at identifying science issues or using knowledge about science. The United States wasn’t particularly impressive at either.
“In every aspect, the challenge [for the U.S.] is getting deeper to the next level of knowledge,” said Andreas Schleicher, the head of the indicators and analysis division of the OECD Directorate for Education, who was presenting.
There was a lot of chatter about Finland, which doesn’t really use standardized tests for accountability, has an extremely strong teaching force, and experiences little variation between the best and the worst schools.
But one interesting thing happened when a woman named Sirkku Kupianinen, a researcher with the Center for Education Assessment at the University of Helsinki who was serving on the panel responding to Mr. Schleicher’s comments, gave her remarks.
Ms. Kupianinen said she felt awkward at all the attention her country’s been given, particularly since Finland’s system runs almost entirely on trust and is nearly devoid of the external accountability benchmarks used in other countries “I feel like the whole country has been raised to a miracle based on the results of this one test,” she said about PISA.
What’s more, despite Finland’s strong showing in math on the test, academics in her country have been raising some fairly strong concerns about the level of math education among students. “They say it’s going down like the tail of a cow,” she said. (Really, she did say that, and man, what a great expression. I’m officially appropriating it.) “Then PISA comes out, and math professors just stopped believing in PISA as ... a measure of what Finnish children can do,” she said.
And her concern? That countries will start trying to encourage “teaching to the test” for PISA, by modifying curricula and so forth to resemble that test’s tasks, which require students to synthesize knowledge. Publishers in Germany, she said, have already released books of “PISA-like” items.
Hmm, teaching to the test, fear of one test serving as the determinant of quality. Where have I heard this before?
Apparently international benchmarking carries its own set of challenges. Some food for thought for Education Secretary Arne Duncan and the folks working on the common core/common assessment.