skoolboy Goes to the Olympics
skoolboy has always found Olympic medal counts by country to be silly. Sure, it's fine to take pride in the accomplisments of one's countrymen and countrywomen. But the Olympics for me are about appreciating excellence, regardless of the flag (or swoosh) on the uniform.
Ah, but student achievement! That's a horse race of a different color. We have a venerable tradition dating back at least to A Nation at Risk of comparing the academic achievement of U.S. schoolchildren to the performance of kids in other countries. The Olympics serves as a quadrennial site for seeing how we measure up to other countries.
Yesterday, eduwonkette decried former West Virginia governor Bob Wise's comparison of the relative performance of elite U.S. athletes against the world in the Olympics with the relative performance of average U.S. students against the world in high school graduation rates. Aren't our elite students doing just as well as those in other countries?, she asked.
skoolboy doesn't have the performance of elite students cued up for comparison, but here are some data from the 2006 Programme for International Student Assessment (PISA), an international survey of 15-year-olds in 57 countries. The figures below show the performance achieved by students at the 95th percentile--that is, the top 5%--in each country. The countries are arrayed from lowest achievement to highest on the PISA assessments, with each column representing a country. Dark blue columns are countries scoring significantly higher than the U.S. Grey columns are statistically indistinguishable from U.S. performance, and bright red columns are countries doing worse than the U.S. The length of the column represents how far away a country is from the U.S. based on the standard deviation of individual scores around the world.
In mathematics, the performance of top U.S. students is dismal. In 28 countries, students at the 95th percentile score significantly higher than students at the 95th percentile in the U.S., and the gaps are surprisingly large. Students in Chinese Taipei, Korea, Hong Kong, Switzerland, Finland, Belgium, the Czech Republic and Liechtenstein all score at least .5 standard deviations above the U.S. in this comparison.
Things look a little bit brighter in science achievement. Ten countries have students at the 95th percentile scoring higher than the U.S., and 35 countries have students at this level scoring significantly worse than U.S. students at the 95th percentile. Eleven countries are statistically indistinguishable from the U.S. Still, the best that we can claim is that the U.S. is tied for 11th internationally, although the magnitude of the gap between U.S. elite students and elite students in the top-ranked countries (e.g., Finland, New Zealand, the United Kingdom, Australia and Japan) is smaller in science than it is in math.
These comparisons don't address the performance of students entering the most selective of U.S. colleges and universities--the MIT's, Cal Techs, Harvards, Princetons and Yales. In a national cohort of 3 million 15-year-olds, the top 5% is 150,000 students, and the vast majority of these are not entering the most selective colleges. Still, the fact that the top 5% of U.S. students are getting their butts kicked in math and science is alarming to those who tie U.S. global competitiveness to the academic performance of American youth. Just as in sport, there are no quick fixes: a well-planned training regimen (including plenty of time in the academic weight room) is the key to success.


Comments
Hi skoolboy,
Thanks for bringing some data to bear on this question - I have to say that I thought we were doing a little better at the top than we are, so I'm glad to get my facts straight.
Though I know you don't need the extra work, I'd also love to see these figures for boys and then for girls - maybe a second Olympic "event" for next week?
Posted by: eduwonkette | August 8, 2008 8:33 AM
Are the students in the better performing countries as well-rounded (no pun intended) as their U.S counterparts?
Posted by: GP | August 8, 2008 8:57 AM
Is the data broken down by race as well?
Breaking the scores down by race would most likely decrease the differential a bit as well.
Posted by: KDeRosa | August 8, 2008 9:02 AM
I am not up on detailed practices in all of the countries that are smoking us at the top--but I know that Finland eliminated tracking some years back, so their overall success tends to be more egalitarian. I am also aware that many countries internationally have gone ahead of us in inclusion of students with disabilities (including Finland).
I don't know exactly how GP would define "well-rounded," but Finland systematically includes the arts at a young age--although not in what we would consider to be "school" (more like after-school programs that all students attend). In Japan, the "shadow" system--which some think of as "cram" schools, actually functions to offer enrichment to most young students (with a more academic bent as high school levels are approached). Many of countries the countries mentioned include languages at a very early (elementary) age. In Finland instruction begins with "mother-tongue" (Finnish, Swedish, Sami, or the homeland language of immigrants), with the addition of a second national language soon after. I would call this evidence of "roundedness."
Posted by: Margo/Mom | August 8, 2008 9:28 AM
KDerosa--the top five percent is the top five percent. How would you break that out by race?
Posted by: Margo/Mom | August 8, 2008 9:30 AM
You could take top 5% of Asians, top 5% of whites, top 5% of blacks.
Alternately you could examine the fraction of each group represented in the top 5%.
Posted by: KDeRosa | August 8, 2008 9:35 AM
Poor skoolboy - we are giving you too much to do!
I also just want to suggest that we think about how these countries' out-of-school contexts bear on their success rather than just focusing on the differences in their education systems. Many of these countries have less overall inequality, broader supports for kids and families, universal health care, day care, etc.
Posted by: eduwonkette | August 8, 2008 9:57 AM
And let's not forget that despite all those things that the average poor person in the US has more income that the average European (poor and un-poor) and far more income than the average asian. And this income does not include quite a bit of non-cash subsidies that we give.
Posted by: KDeRosa | August 8, 2008 10:11 AM
KDR--cherry-picking by race to get the kids who get the most out of education only makes use of our weakness of maldistribution to make us look good. While we are at it, why not eliminate those with low income, males, or anyone else who might make us look bad. Of course, then we would have to do the same thing for the countries we compare to. Take the burakumin from the Japanese, the girls from the countries where they don't do as well, immigrants (except where they excell), etc.
Not entirely sure what point you were making in the assertion that poor people in the US are better off. You might also want to read some of Pong's work (someone to know a few days back) regarding how the US stacks up against other countries in aid/policies that impact those on the economic bottom and the resultant impact on equity in academic achievement. Suffice it to say, we are no international model there either.
Posted by: Margo/Mom | August 8, 2008 10:38 AM
"our weakness of maldistribution"
What does that mean? It sounds very Marxist. Are you saying we don't distribute education well or at least not as well as they do in foreign countries? The correlation between education spending and student achievement is very low in the U.S. and most big city schools are funded above the median funding level in most states.
I don't want to cherry pick data, I want to see disaggregated data by race. A fair comparison would be to compare whites in America to whites in Europe since these groups are genetically similar and to compare Asian-Americans to Asians in Asian countries. In order to determine which countries have better education systems it would be helpful to control for student effects.
With respect to Pong's work you are confusing correlation with causation. The jury on whether those correlations actually cause the things you think they cause is still out.
Posted by: KDeRosa | August 8, 2008 11:20 AM
KDR:
"Are you saying we don't distribute education well or at least not as well as they do in foreign countries?" BINGO! at least in regard to some foreign countries whose top scholars are doing better than our top scholars. But go ahead and compare American African-Americans to Finnish African-Americans. I would love to see the results.
Regarding Pong's work--yes, I was looking at causality, but also at the shocking difference between supports available for children in other countries (child benefits, maternity leave, parental leave) compared to this one, the additional tax-burden laid on single parent families in this country--and despite this, the differences in percentage of single parent families, births to to mothers under 20, divorce rates.
I don't see our deep-seated abhorance for social welfare policies changing much in the near future. But I do think its time we at least entertain the notion that we're doing it wrong.
Posted by: Margo/Mom | August 8, 2008 11:47 AM
If we're talking about international competitiveness, it's not simply white American kids competing against white European kids, and Asian-Americans competing against Asia, but Americans competing against other countries. In the real Olympics we don't dock countries for being on average bigger or smaller than others, or for having a better athletic training system. The best wins, period. The concern with US educational achievement is, I think, a lump sum sort of thing. Whoever it is you are counting, the top 5% of our students, on the tests skoolboy cited, aren't doing as well as the top 5% of many other countries. That's cause for alarm, even if, say, Asian-Americans are comparable to Singaporean Asians.
Less importantly, I'm also not convinced that comparing, say, white Americans to white Europeans, or Asian-Americans to Asians, is terribly fair. Putting my discomfort with genetic determinism aside, the group "Asian-Americans" are only but so close to the group "Singaporeans" or "Koreans". And beyond that there are all sorts of cultural differences - wealth (as you mentioned), minority vs. majority, being an immigrant or descended from immigrants vs. not being either, cultural differences beyond that of schooling.
Basically I don't think you can control for student effects when looking at international data, nor do I think you should. The global economy doesn't control for effects; why should we?
Posted by: MS | August 8, 2008 12:00 PM
I wouldn't presume to curtail debates about race and ethnicity, but when it comes to the PISA data, the only thing available AFAIK is race/ethnic data for the U.S., and we already know that story. If I can muster the energy, I will try early next week to summarize data on (a) high scorers by gender in different countries, and (b) cross-national variation in the strength of the SES-achievement relationship.
Posted by: skoolboy | August 8, 2008 12:08 PM
A great case study that adds to Skoolboy's analysis:
The Bellevue School District in Washington is a very high performing suburban school district with multiple high schools being ranked in the top 100 by Newsweek. They have 90% of their students going to college. Great SAT scores etc...
But in 2004, BSD analysed the performance of all their students using the TIMSS format and found substantial underperformance of their students.
In analyzing student performance, William Schmidt concluded that for math,
"The performance level of Bellevue's eleventh and twelfth grade students was equal to eighth grade students in the composite of top acheiving countries."
bsd405.org/portals/0/aboutBSD/TIMSSBellevueReport.pdf
(The composite countries were Singapore, Korea, Japan, Hong Kong, Flemish speaking Belgium, and the Czech Republic)
So our best schools are giving our top students what would be considered, internationally, an average eighth grade math education.
Our students, impoverished and well-off, are all being greatly underserved by our school system.
Posted by: Erin Johnson | August 8, 2008 12:17 PM
KDR writes: "A fair comparison would be to compare whites in America to whites in Europe since these groups are genetically similar and to compare Asian-Americans to Asians in Asian countries." What, precisely does this mean? As world-famous geneticist Craig Venter argues, "race is a social concept, not a scientific one." False notions of genetic difference between races--differences that supposedly affect school performance--have been put to rest by science, thank goodness. So, what exactly is the point of such disaggregated comparisons?
Posted by: CV | August 8, 2008 12:59 PM
CV asks: "So, what exactly is the point of such disaggregated comparisons?"
Only point I can see is to try to bring KDR up to speed with the rest of the scientific community.
Posted by: Margo/Mom | August 8, 2008 1:49 PM
Putting my discomfort with genetic determinism aside
Why put aside a moderate position, which recognizes the influence of both genetics and environment so that you can embrace the radical position of environmental determinism where environment is assumed to be responsible for 100% of variance? Why do you cast yourself as an ideologue and extremist in such matters?
As world-famous geneticist Craig Venter argues, "race is a social concept, not a scientific one." False notions of genetic difference between races--differences that supposedly affect school performance--have been put to rest by science, thank goodness. So, what exactly is the point of such disaggregated comparisons?
As world famous Population Geneticist Armand Leroi writes in the New York Times:
You can be the judge of whether such luminaries as Venter and Collins actually BELIEVE that race is a social concept or whether they, like many in the field, don't want to touch the issue with a ten foot pole for fear of being Watsoned, and instead divert their attention to other research programs and issue the standard squid ink when the question of race comes up. The following account published in Scientific American portrays either incompetence on the part of Collins or his use of squid ink to protect his reputation and funding:
Only point I can see is to try to bring KDR up to speed with the rest of the scientific community.
KDR is more up to speed on this issue than folks who imbibed their knowledge from the likes of Stephen J. Gould, who was so beloved by the literati but who got so much of his science popularizations wrong.
Posted by: TangoMan | August 8, 2008 3:24 PM
Q. Why put aside a moderate position, which recognizes the influence of both genetics and environment so that you can embrace the radical position of environmental determinism where environment is assumed to be responsible for 100% of variance? Why do you cast yourself as an ideologue and extremist in such matters?
A. irony.
Posted by: MS | August 8, 2008 3:39 PM
Margo/Mom, correlations notwithsatnding where is the empircal evidence that supports your assertions that actually improving these social conditions will lead to an increase in student achievement? Where is the large scale replicated study with an effect size greater than 0.25?
Posted by: KDeRosa | August 8, 2008 4:25 PM
Basically I don't think you can control for student effects when looking at international data, nor do I think you should. The global economy doesn't control for effects; why should we?
What's the point of these international comparisons? Mostly, the results are used as benchmarks and to initiate reform. Comparing across populations using Creationist models of human uniformity results in reforms destined for failure. To get meaningful insight and to properly structure reforms one must fully understand the parameters of the analysis - control for differences in populations and culture and focus on matters that are amenable to change.
Posted by: TangoMan | August 9, 2008 5:00 AM
I am curious as to the US's placement over the years. Are we losing ground yearly? When did we place highest?
Posted by: Diane | August 12, 2008 4:04 AM