« NYC Links: Klein Petrilli Barcelona | Main | This Week's COWAbunga Award »

skoolboy Peeks out of the Closet

| 26 Comments
00380018cropped.JPG

spiffboy2-thumb.jpg

Now that eduwonkette has revealed herself as Columbia doctoral student Jennifer Jennings, skoolboy is gingerly sticking his head out of the closet and looking around. (If I see my shadow, I may go back inside for another six weeks.) skoolboy is Aaron Pallas, a Professor of Sociology and Education at Teachers College, Columbia University. I study inequalities that are created and perpetuated by the ways schools sort and select children and youth, and the role that education plays in individuals’ adult lives. Recently, I went on the record in the New York Sun on a topic near and dear to eduwonkette’s heart: the failure of New York City to make substantial progress in reducing the achievement gap among different racial and ethnic groups.

What’s my relationship to eduwonkette? She took a couple of courses with me, and I’m on her dissertation committee. (Her dissertation contrasts the consequences of accountability systems in education and medicine. A provocative entry into the topic is her Ed Week commentary, under her own name, here.) More importantly, though, we’ve been collaborating on a series of studies that look at the mechanisms by which some New York City schools garner more resources than others. All of the qualities that make her blog compelling and so much fun are just as evident in her approach to academic research.

eduwonkette said the other day that she stands behind everything she wrote under the pseudonym. I do too, on substance, but I’m not as sure about tone. I think the conventions of blogging, especially anonymously, allow for shooting from the hip quite easily, and my usual writing is more painstaking. (More long-winded than my interminable dreary posts? Yep.) Also, I think that sometimes I try to emulate eduwonkette’s style, which is appealing—and she is expert at it—but I’m not skilled enough to pull it off. So if I’ve offended anyone through my tone, either in the past or in the future, my apologies.

Finally, unlike eduwonkette, I did become an academic to talk to five guys in a room with transparencies. Only now, we use PowerPoint.

And I’m so glad it’s no longer just guys.

Apologies to Ed Week: earlier, I said that only subscribers could get to eduwonkette's Ed Week article on accountability and risk adjustment in education and medicine. You can get to it through the link above. eduwonkette's better at the technology than I am, too. And while I have the floor: Thanks, Ed Week, for giving eduwonkette the space to create such an interesting forum for discussions of education research and policy.

26 Comments

Skoolboy: Nice to meet you. As a DC area resident, is anyone in the education blogosphere NOT in New York?

Attorney DC:

I'm a DC resident and I blog about English-language learners for EdWeek at Learning the Language at www.edweek.org/go/ltl/. So not everyone who has an education blog lives in New York City.

The group of kids I write about isn't exactly on everyone's radar screen. I'm really pleased whenever I see any of the education bloggers draw attention to them, which is kind of rare.

Mary Ann Zehr

Skoolboy,

I study inequalities that are created and perpetuated by the ways schools sort and select children and youth, and the role that education plays in individuals’ adult lives.

I for one would welcome some blog posts on:

a.) Inequalities that are created by school procedures;

b.) Inequalities that are perpetuated by school procedures and how you determined that perpetuation is at work;

c.) The effects that sorting and selecting of children have on measurement of inequality;

d.) and since you described the role of education in adult lives in such a general fashion, I'd be very interested in your specific thoughts on this issue.

"We need to be aware that what we're doing right now to close the achievement gap may not be working," Mr. Pallas said. "If what we're doing isn't working, we need to be aware of that and perhaps think about doing something else."

This quote from the NY Sun article begs an obvious question - what exactly should they be doing that will work? The only method that I'm aware of that yields measurable results is to increase time on task by about 50%. Are there any shortcuts that haven't failed over the last 50 years but are simply not finding an audience that will add them to their teaching toolkit?

TangoMan,

My point in the New York Sun article was not that there are great solutions out there that are being overlooked--because I don't think that there are--but rather that it would be unfortunate if local education agencies emulated New York's reforms expecting to see a substantial closing of the achievement gap. Because the evidence is quite clear: in spite of the grand claims made by Mayor Bloomberg and Chancellor Klein, that hasn't happened in New York.

The reality is that most social and educational programs don't "work," in the sense of living up to the hopes and expectations of those who design and champion them. But it's much easier to explain why a program didn't work than it is to demonstrate what will.

The reality is that most social and educational programs don't "work," in the sense of living up to the hopes and expectations of those who design and champion them.

I think that you offer a very accurate summation of the dynamic but in doing so you highlight the achille's heel of the education profession - why exactly are these champions putting the cart before the horse and advocating policies and programs that haven't been validated to deliver results and replicated by other researchers. Imagine if the FDA worked in that fashion - the drug designers would market their drugs and then after large scale market capture the public would discover that the whole effort was a sham and that the drugs didn't deliver the benefits that were promised. How many people would be going to jail for fraud? How much money would have been wasted? How many lives would have been disrupted?

Implementing school policies that simply sound good, but haven't been validated, which consequently fail to deliver promised results after implementation, yields the same types of outcomes mentioned in the drug example - these programs waste money, they disrupt lives, and in fact many of them cause actual harm to children. Why isn't the education establishment insisting that snake-oil tactics be disavowed and standard social science benchmarks be adopted?

Re Powerpoint - If I remember correctly, I've been stuck flipping transparencies at AERA for at least 3 of the last 4 years.

Let me take this opportunity to thank skoolboy for being the most amazing blog collaborator one could ever hope for - looking forward to many more of your blog posts!

Wonder now that you and Jennifer have revealed yourselves you could write something more about your hesitation in doing so initially, the pressures that keep other academics quiet rather than criticize the administration's policies, and why the two of you changed your minds.

thanks,

Aaron and Jennifer,

As a parent of children who suffer or benefit as we speak from the impact of curricula like Everyday Math, Balanced Literacy and TCs own Writer's Workshop, TangoMan's comments hit the nail on the head.

By way of example the IES's What Works Clearinghouse (http://ies.ed.gov/ncee/wwc/) lists 74 elementary math curricula, of which 5 have enough peer-reviewed data upon which to draw a conclusion about effectiveness or impact. Seven percent! And remember, that's not that they're proven effective or ineffective; it's just that we have data to study them.

If this were Tonka or Mattel, the CPC would be all over the textbook publishers. In deciding to adopt "Everyday Math" I suppose it doesn't really matter that there are just 4 studies out of 61 that are valid enough to study for signs of effectiveness. But even those four studies are only moderately positive, with just one study finding a statistically significant positive impact

As the Chancellor would say "Where's the Outrage?" I guess if it's only about "statistical significance" it's just a game?

Shonda, as Abbie Hoffman once said.

Lots of excellent questions from TangoMan and Matthew about why we seem to have less decisive evidence on the effectiveness of interventions in education than we do in medicine. This warrants a longer response than I can give here, but as a first step, I'd like to encourage readers to look at a paper by my colleague Carolyn Riehl entitled "Feeling Better: A Comparison of Medical Research and Education Research," published in the June/July 2006 issue of Educational Researcher. Riehl, yet another cool person you should know--anyone who collaborates with me is, by definition--points out that we frequently assume that the connections between randomized clinical trials and other forms of evidence to medical practice are radically different than these connections are in education. Instead, she argues, there are many similarities, in spite of the fact that medical research is much better funded and organized.

Lots of excellent questions from TangoMan and Matthew about why we seem to have less decisive evidence on the effectiveness of interventions in education than we do in medicine.

Here's my hypothesis - teachers don't think like scientists, they're more idealists at heart. They envision a certain role for themselves and they gravitate to approaches that reinforce their idealism.

We don't see the same sense of idealism over-riding evidence-based reasoning in the medical profession, the accounting profession, the engineering profession, the drug research profession, etc. Look at the resistance to script-based teaching approaches. I venture that much of this resistance isn't based on appeals to evidence that script-based teaching is not effective, rather the rejection is based more on the diminishment of the teacher's unique influence over their young charges.

Secondly, the action of "doing something" is preferable to waiting for valid methods to develop, especially when there is a presumption that teachers and methods are all that stand between equal educational outcomes and wide-ranging gaps in performance. The fact that this model of how things works is divergent from reality is little consequence to idealists, for they're driven, when push comes to shove, by belief, not evidence, and that's why they continue to believe, even with little supporting evidence, that the solution is just around the corner and will eventually be found and can then be easily implemented.

Further, it's assumed that no harm is done by implementing invalid methods because the teacher doesn't mean to cause harm, as though good intention will inoculate the students from bad practices.

Evidence-based reasoning is very highly stressed to students of science, engineering, and medicine but enjoys far less prominence in the education schools, where instead we see more emphasis on "theory formation" and celebration of "theory." IMO this is why "theories" are rushed out as gospel, mostly having bypassed stages of hypothesis formation, falsification efforts, replication efforts, validity checks, etc that lead to theories gaining currency and there are too few skeptics who can stand in the way of stopping emergent fads. Further, too much education research is devised and implemented by the individual researcher or a small team, rather than having different groups devise the hypothesis, gather the data, and analyze the data, so we find published work that is more prone to being tailored towards desired outcomes.

Any solution has to start at ground zero, that is, the point where all teachers find common origin, education schools. These institutions must inculcate skepticism into the practitioners of teaching and those who make careers of research. The two writers of this blog set a good example, especially with the NYC Dept of Education data and pronouncements. More teachers need to adopt the practice of skepticism and chuck overboard the role of advocate for approaches that appeal to them and the role of progressive educator who is intent on implementing the cutting edge of new approaches (where more emphasis is given to the notion of progress than efficacy.) They need to learn to look at a new approach and find that their first instinct should be to tear it apart, rather than to embrace the approach and try to give it a chance. The latter approach is very nurturing while the former is very confrontational, yet most of those involved in evidence-based endeavors will argue that more good comes from a model which survives a confrontational approach than comes from the nurturing of models to the point of uncontested failure for scarce resources are not wasted in nurturing a faulty model past obstacles that should, under the confrontational approach, immediately invalidate the model, thus freeing up resources for other models to be tested. To put it simply, less embracing and more skepticism needs to be at the core of the education school experience.

Skoolboy - Nice to finally meet you - online. Leonie's request for some thoughts on your and eduwonkette's collaboration and the issues related to academics blogging would make interesting reading. It seems your support of her after initial hesitation was important to the process and success of this blog.

That it took less than a year for you guys to reveal yourselves may be one of the major successes of this blog - opening up the process to others.


It took me a bit of time to accept that there were problems with blogging in the academic world.

I didn't take it as seriously as I should have at first, eduwonkette convinced me over time that there is a considerable minefield for a grad student. You guys still may be unique but with all the attacks onyour being anonymous I still think,just as it is for so many teacher bloggers who talk about their schools, students and colleagues, anonymity is still a valid option, notwithstanding the attacks.

TangoMan,

Education is not like medicine. Medicine is for something that is specifically wrong and the treatment can be shown to either help or not. Education is more effemeral and long term. We even have a great deal of difficulty even defining what a good education is.

But going with the medicine analogy. Doctors are practitioners. It is very rare when the clinicians that treat patients come up with their own treatments. New treatments are for the most part developed by scientists or companies that have a financial motivation to produce something that works. And nothing is allowed for public use until the FDA approves it. We have a process in place to evaluate the evidence *before* doctors are allowed to prescribe or use the procedure. And even after that, highly experienced doctors publish best practices summaries.

It is only after the evidence has been vetted that the doctors will be allowed to use it. The doctors themselves are not allowed to cicumvent that process or risk malpractice lawsuits.

Even if education schools stressed more skeptism and evidence, there is little/no published evidence on any methods.

While our education schools could do substantially better, changing teachers to just be skeptical will do nothing to improve the education that our chidren receive because it does not provide *any* process for improving schooling.

We put to much on the backs of our teachers. We expect them to be everything; invent something new, connect with our children, follow every fed, state and district mandate and still have sometime to teach.

It is our system (or lack there of) of developing quality curricula and assessing whether a particular approach works well with our children that is completely missing. And because this is practical, applied research this type of work can only be done within the school themselves.

Even if education schools stressed more skeptism and evidence, there is little/no published evidence on any methods.

From my vantage point that's a pretty astute observation. It's also fairly damning towards the professional standards embraced by education faculties.

I wrote my comment late at night and didn't put too much thought into all of the issues that should have been covered so teachers really aren't my prime target for criticism, rather I'm aiming fire at the profession as whole, which includes the researchers. As you correctly point out the teachers are the practitioners and they must rely on the researchers to provide them with the intellectual tools to use in the classrooms.

Too often the standards of professional research and graduate class content are what appears to be a dismal joke. I wrote of this article published in the Teachers College Record back in 2004

Movements of Mind: The Matrix, Metaphors, and Re-imagining Education Alison Cook-Sather | 2004

This article uses the popular move The Matrix to evoke both metaphors for human existence and models for teaching and learning, it examines two metaphors that have dominated notions of and approaches to education in the United States, and it argues for seeking, crafting, and embracing metaphors that cast students as the principle creators of their education and themselves.

Here I write of the course design for a graduate class on multicultural math.

In all fairness, it's improper to lay all the blame on teachers for they have to run the gauntlet of hip progressive "content" in order to get in front of a class of students.

I've argued on other education blogs that the system needs to be redesigned from the ground up and I'd be hard pressed to argue a case in favor of ed schools deserving their current stature in a reformed system. There are too many stories like this one, from a Columbia student:

My instructor then interrupted and told us we weren't going "deep enough." He said, "what makes families and communities the way they are?" He was trying to make us talk about societal structures of oppression. Again. We never talk about teaching, we always have to talk about the societal structures of oppression.

TangoMan,

We would both agree on the lack of scholarship of many "researchers" in ed schools.

Even if you changed all of the ed schools to have great research and wonderful preparation it would make little difference in the schools themselves.

The fact that ed schools can get away with such nonsense is more a reflection of the lack of demand for quality instruction by the schools themselves.

Schools are not organized to decide what a quality education(s) might look like, evaluate whether children received that quality eduation and what aspects of teaching and curricula are needed for success.

The excesses of ed school make for a very easy target. But to improve our schools, we need to look at the schools themselves. In particular, how do schools improve? How do they improve curricula? Who in the schools sets long term goals for their students? How do they develop assessments? Are the assessments both alighed with classroom instruction and the long term goals? And most importantly, how do they know that they are doing a good job? etc... It is answering these questions that we will get to improvements in our schools. Not by blaming ed schools.

Skoolboy,

Regarding your comment: "My point in the New York Sun article was not that there are great solutions out there that are being overlooked--because I don't think that there are--"

If there are not great overlooked solutions, then how is that successful school systems around the world ( Finland, Macao-China, etc...) are able to raise the achievement of all their students to levels substantially higher than the US while simultaneously decreasing the achievement gap due to SES or immigrant status?

There are great overlooked solutions to our schooling problems in how quality, sucessful school systems operate around the world. But given our myopic, US-centered focus of "improvements" (of which there has been almost nothing over the past 30+ years), these solutions are neither popular nor are they being discussed in the serious manner that might result in actually improving our children's education.

Erin:

Fair enough. To be more precise, I might have said that I don't think that there are any great solutions out there that are politically feasible in light of the longstanding tensions among the diverse purposes of U.S. schooling. I think it's nigh impossible to think about school reform in the U.S. independent of the political, social and economic forces and ideas that have brought us to where we are today.

If there are not great overlooked solutions, then how is that successful school systems around the world ( Finland, Macao-China, etc...) are able to raise the achievement of all their students to levels substantially higher than the US while simultaneously decreasing the achievement gap due to SES or immigrant status?

It's in situations like this that I would have preferred skoolboy and eduwonkette to remain anonymous, because they can't even moot some points without fear of being Watsoned. I'm not trying to speak for them, don't get me wrong, all I'm saying is that they've lost some degree of freedom by coming out of the closet.

As to your point, the homogeneity of the population in Macau or Finland is extremely high. If you want to compare them to the US, I would suggest comparing them to a county of similar size, or a State of similar size. Look at Westchester County and compare to Macau. Even there Macau is still more homogeneous than Westchester.

There is a growing body of literature which is documenting the negative effects that come along with diversity. No methods have yet been devised that can ameliorate these effects. Here too we see the normative vs. prescriptive aspect to policy formation. The desire to seek diversity is mostly based on normative considerations (this will make the world better according to the philosophical worldview many hold) yet the data shows substantial consequences.

Take a look at Finland's immigration system and their level of ethnic diversity. All Finnish citizens are rooted into their broad culture, unlike the US with its multitude of cultures. Expectations and behaviors are constrained by narrower cultural boundaries than is the case in the US.

higher than the US while simultaneously decreasing the achievement gap due to SES or immigrant status?

I'd like to see the data that you're using as the basis for the comment that they're decreasing the achievement gap associated with immigrants or low SES.

Skoolboy, I can see why you think that but disagree. The key elements that enable those school systems to do well could easily translate into US culture. But the discussion on ed reform would have to start with why those systems work well and how that could translate instead of pointing out all the reasons why not.

TangoMan, Macao is ~20% immigrants and ~60% 1st generation. Hardly considered uniform. And many of the immigrants come from very impoverished backgrounds, so the number of students that would be considered low SES is rather high. And yet their contribution from SES is less than 3%.

The data comes from PISA (2003 is the best because the US made an implementation error on the 2006)
www.oecd.org/dataoecd/1/60/34002216.pdf ~pg165

Many other countries have similar immigration and SES distribution to the US. It is a mistaken idea that the US is unique in its SES and immigration challenges. The high quality school systems greatly diminish this contribution. Internationally, the best systems have less than 10% contribution from SES where the US has about ~20%.

If you wish to take the US out of the picture, one of the most illuminating comparisons between school systems is the German system and the Finnish system. Culturally, the two countries are almost the same. But educationally, the Finns dominate. The German system has a tracked system and the Finns educate all their students to a high level.

One might think that by tracking their high performing students and challenging them, those German students would outscore the Finnish ones by a large margin. But that doesn't happen. The high-tracked Germans (not their immigrants or low ability students) score just the same as ALL of the Finnish students (high ability and low ability). School systems matter. And they matter a lot.

In analyzing the PISA data using individual student scores, Woessmann demonstrated that ~25% of the variation in student acheivement was due to the type of school system used in the country. The school systems that were successful were characterized by school systems that use external exams (little/no teacher grading) with school autonomy in selecting textbooks and in school budget allocation.
www.springerlink.com/content/x0848k4148w76314/

We could improve our children's education substantially and rather easily, if only we were looking in the right places.

TangoMan, Macao is ~20% immigrants and ~60% 1st generation.

Macao, with a population of 500,000, has a population which is comprised of 95% Chinese, the remainder are primarily either Portuguese or mixed Chinese-Portuguese ancestry.

Migrating from Suffolk County (population 1,419,000) to Nassau County (population 1,355,000) is a the type of migration that characterizes Macao. If they had the type of migration the US experiences, from different countries and continents, then their example might be comparable to our experience.

And many of the immigrants come from very impoverished backgrounds, so the number of students that would be considered low SES is rather high.

They are not getting Portuguese or mixed Chinese-Portuguese migrants, for those are the folks who have deep roots in Macao. Their migrants are Chinese. Their migrants are drawn from a fairly homogeneous population. Their low SES is mostly immaterial because a.) their political/economic system has over the generations suppressed most people into a low standard of living; b.) the system hasn't been characterized by wide scale assortative mating; and c.)the SES --> student performance correlation isn't as powerful as you believe absent cofactors.

The high-tracked Germans (not their immigrants or low ability students) score just the same as ALL of the Finnish students (high ability and low ability). School systems matter. And they matter a lot.

Germany is located in the center of Europe, Finland on the periphery. From a population geneticist's point of view Germany is going to have far more admixture from neighboring populations than will Finland. That is, those who are born German have far more genetic variety in their population group than those who are born as Finns. Now throw onto this biological foundation, the wreckage instilled by East German policies and the cultural developments that resulted and you add mightily to the variation seen in the dependent variable. Finland is a small, cohesive culture, and the population is 97.6% Finnish, with the minority groups being 0.6% Russian, 0.12% Roma, 0.11 Sami. Cultural homogeneity has a powerful effect on establishing norms of behavior whose effects are not factored into the "school practices" analysis.

Woessmann demonstrated that ~25% of the variation in student acheivement was due to the type of school system used in the country.

So long as sociologists are education theorists are operational creationists, their analysis will always be flawed. Woessmann didn't control for mean level of national IQ and he didn't control for cultural homogeneity, so his study suffers from poor model design.

To illustrate my point in an exaggerated fashion, let's study two systems of education - the first is being implemented for a normal range of kids and the second for kids with developmental difficulties. If we don't control for cognitive ability then the variation attributed to mode of instruction will be far larger than it truly is. Culture and genetics matter. To ignore them and pretend that mode of instruction is the most crucial hypothesized factor is to engage in a flawed study.

We could improve our children's education substantially and rather easily, if only we were looking in the right places.

No we can't. However, I would actively back any calls that would put your hypothesis to the test so long as we could agree that the hypothesis would be abandoned if it was falsified.

TangoMan,

Now if either Suffolk or Nassau county could enable their students to learn as well as Macao-China with statistically higher learning than the US as a whole coupled with an SES contribution of less than 3%; then that would be something to be proud of.

Woessmann's model accounted for ~85% of the variance so certainly other factors could come into play. A 25% variance due to school systems effects across 41 countries would require an extra-ordinary IQ contribution. If you believe that IQ confounds the analysis based upon individual student scores in 41 countries, what would the IQ differential effect need to be to invalidate their conclusion? Certainly, my quick calculations suggest something that is biologically impossible.

Perhaps if you don't like the German to Finland comparison, look at the difference between Belgium and/or Netherlands and Germany. Again, two cultures that have great similarities in cultural norms, homogeneity, genetic backgrounds and immigrant status. And yet the acheivement differences between either Belgium and/or Netherlands compared with Germany are statistically higher. Are you trying to suggest that the Germans are more inherently stupid than the Dutch or the Belgians?

If you are looking for ways to validate your ideas that it is impossible for our schools to improve (IQ, SES, cultural traditions, etc...), then you will surely find what you seek.

But the international evidence does not support your position.

Now if either Suffolk or Nassau county could enable their students to learn as well as Macao-China with statistically higher learning than the US as a whole coupled with an SES contribution of less than 3%; then that would be something to be proud of.

1.) Why are you comparing a student population in the US which is comprised of whites, blacks, jews, asians, native english speakers, non english speakers, to a student population in Macao that is almost 100% Chinese?

I'm not inclined to go digging around for data that can compare Macao's student performance to Asian-American student performance, but you might be interested in digging it up. I'd suggest looking for schools that are nearly 100% Asian in student body. None come to mind right away, but I know of some schools in CA that are in the 50%-70% range. Try Mission San Jose High School which is 77% Asian. My point here is that you need to compare like to like. Asian students in Macao with Asian students in the US. Next up, you need to control for their cultural environment as best you can. Asian student in the US are quite highly dispersed so rarely do they form the dominant group in schools or communities.

2.) Indulge me by sticking with me on this long winded discourse. Let me set up an analogy to aid in my effort of eventually getting to a point. :) Let's look at the genetic-environmental influence on skin cancer. We take a large cross section of society and we note that some particular people are more prone to skin cancer than others. After some study we conclude that genetics plays a significant role, skin pigmentation as a first order effect, in the onset of cancer. For some reason however this conclusion is deemed insufficient and we want to downgrade the influence of genetics, let's say the Sunscreen Manufacturers fund the study because they hope to show that ALL PEOPLE have equal need to use sunscreen and they want the sales of their products to increase. So we separate out the fair skinned redheads and blondes from the main group which encompassed peoples from all racial backgrounds because we hypothesize that this group, which has a disproportionate incidence of skin cancer, may have distinct behavioral traits that increase their risk for developing skin cancer. Now, within this restricted range of subjects we also parse by people who've had family history of skin cancer and we chose to study only them. Within this group of subjects there are office workers and there are people who work in the outdoors. There are people who use sunscreen religiously and cover themselves while others do not use sunscreen and don't take any particular care about shielding themselves from the sun. So, what will we expect to find? Of course, we'll see that the behavior has an extremely large role to play in whether one develops skin cancer and that genetics plays a diminished role for all of the subjects were drawn from a restricted range of subjects, namely those who were fair skinned and had a family history of skin cancer. We've restricted the genetic range considerably and thus increased the influence of environmental factors.

Now here's my point - you draw our attention to Macao's record on education and the fact that there is a "SES contribution of less than 3%." What is the SES contribution in the US? Considerably higher. We should ask ourselves why that is. What's unacknowledged here is that the study that found Macao's SES contribution to be 3% has inadvertently controlled for genetics, the narrow range of SES strata, and the consequent diminishment of the influence of assortative mating on life outcomes, which American studies do not even acknowledge as phenomena that exist. In the skin cancer example, once you restrict the range of your study, you decrease the influence of the excluded heterogeneous variables. Now in this case, the fact that SES has such a weak effect should be telling you something startling about the effect of SES in the American mileau, where we don't experience a homogeneous population, high cultural uniformity, a narrow band of SES strata, and where assortative mating is fairly pronounced.

Woessmann's model accounted for ~85% of the variance so certainly other factors could come into play.

If Woessman models the following:

Parent's SES ---> child's achievement

but completely neglects a more detailed model,

Parent's IQ --> Parent's SES --> child's IQ --> child's achievement

then Woessman will be capturing a dynamic that is in play but there will be gross misattribution of influence involved for the independent variable is not really Parent's SES but Parent's IQ. Secondly, beyond IQ are other heritable traits that parent's pass onto their children which have marked impact on the child's performance in school and in life. These other traits enjoy significant correlations with SES, and again, SES is the dependent variable, not the independent variable.

A 25% variance due to school systems effects across 41 countries would require an extra-ordinary IQ contribution.

The APA notes that the correlation between IQ and school performance is 0.50.

Look, as I noted in my other comment, I'd be a strong voice in favor of a no-holds barred push to implement all of these favored approaches as pilot projects so long as all of the participants would give up excuse making if they failed to live up to projections. Further, if you can demonstrate that by adopting the methods you favor that you can transform an inner city school with 100% black student attendance or a school in an upper class Black neighborhood into schools that perform to identical standards as schools in Macao, with 100% Chinese student body, or Finland, with 100% Finnish student body, then I will celebrate your achievement and rethink where I went wrong with my analysis. The fact that this has never been done, though parts of it have been tried through international cross fertilization of best practices, is highly suggestive of the fact that it can't be done.

TangoMan,

Even if you substituted IQ for SES contribution, the variablity due to school structural organization in the Woessmann study is still 25%.

The point of doing cross-national studies analsyses is do the exact opposite of your skin cancer analogy. Woessmann used individual student data from all 41 countries and the countries were spread out over the world.

If you are interested in how a single country could dramatically improve their student's learning, you may consider Singapore. While Singapore has been well known for its great math and science performance, what is less well known is how they dramatically improved their reading instruction and consequently their students' reading ability. (as measured on the PIRLS)

In 2001, Singapore performed on an average level on the PIRLS. Dissatisfied with their students' performance, their MOE designed and implemented a new program for teaching beginning reading.

Their students learning increased dramatically. (standing on the 2006 PIRLS rose 2nd/3rd best nation tested in English.) Remarkable, considering that the tests were given in English, which is a 2nd language to ~75% of the students.

Instruction matters. School organizations matter. Our schools can become better places to learn, if we could recognize that the myopic, US centered view of ed reform is not working.

Instruction matters. School organizations matter.

I'm not saying that instruction or school organization don't matter. I'm saying that gains from adoption of best methods are significantly influenced by the characteristics of the student body. So, the gains seen in Singapore are unlikely to be matched to the gains seen in the LAUSD when the same reforms are implemented.

I'm a bit afraid to jump in here, as I am just a lowly teacher, but am a former critical care RN. You all have touched on some very interesting issues in comparing teaching to the medical model;something I do quite often. Yes, medicine does operate from "previously proved" model. But there are other huge differences that education could benefit from looking at. One is the medicine is prescriptive. First you make a diagnosis, then tailor a treatment plan. Kids don't read "on time" for a variety of reasons, but we are not given the tools, time or personel to make a diagnosis,(though the ones we have available are pretty crude) and then are not given the time, tools or personel to develop and implement a treatment plan.

And you implemenet this treatment plan one patient at a time. You also get to rely on one or two to dozens of others to implement the treatment with you. Teachers - well we get me, myself, and I and an aide for maybe 30 minutes twice a week for my 33 kids, with no diagnostic time cause we're busy treating!

While we have standard algorithms and treatment plans in medicine for specific conditions, we don't apply them without tailoring and continually evaluating. In addition we tolerate a spectrum of outcomes as success. People respond as individuals to medical intervention and success is defined differently for different individuals. If cardiovascualar surgeons had to treat their patients the way we have to "treat" our "educational patients"(only one way, without considering any complicating factors of the presentation, and if it doesn't work, then do more of the same for more minutes in the day) they'd all be low performing also - there'd be a lot of dead folks!

Sigh... but here is how it works.
Me, "we should only use high-quality experimental studies."
Legislator, "what do they say about ...?"
Me, "Nothing, we don't have any."
Legislator, "Seriously, you want me to go in with 'Who knows?'"

Comments are now closed for this post.

Advertisement

Recent Comments

  • edpolicyguy: Sigh... but here is how it works. Me, "we should read more
  • carol: I'm a bit afraid to jump in here, as I read more
  • TangoMan: Instruction matters. School organizations matter. I'm not saying that instruction read more
  • Erin Johnson: TangoMan, Even if you substituted IQ for SES contribution, the read more
  • TangoMan: Now if either Suffolk or Nassau county could enable their read more

Archives

Categories

Technorati

Technorati search

» Blogs that link here

Tags

8th grade retention
Fordham Foundation
The New Teacher Project
Tim Daly
absent teacher reserve
absent teacher reserve

accountability
accountability in Texas
accountability systems in education
achievement gap
achievement gap in New York City
acting white
admissions
AERA
AERA annual meetings
AERA conference
AERJ
Alexander Russo
Algebra II
American Association of University Women
American Education Research Associatio
American Education Research Association
American Educational Research Journal
American Federation of Teachers
Andrew Ho
Art Siebens
ATR
Baltimore City Public Schools
Barack Obama
Bill Ayers
black-white achievement gap
books
books on educational research
boy crisis
brain-based education
Brian Jacob
bubble kids
Building on the Basics
Cambridge Education
carnival of education
Caroline Hoxby
Caroline Hoxby charter schools
cell phone plan
charter schools
Checker Finn
Chicago
Chicago shooting
Chicago violence
Chris Cerf
class size
Coby Loup
college access
cool people you should know
credit recovery
curriculum narrowing
D3M
Dan Willingham
data driven
data-driven decision making
data-driven decision-making
David Cantor
DC
Dean Millot
demographics of schoolchildren
Department of Assessment and Accountability
Department of Education budget
desegregation
Diplomas Count
disadvantages of elite education
do schools matter
Doug Ready
Doug Staiger
dropout factories
dropout rate
dropouts
education books
education policy
education policy thinktanks
educational equity
educational research
educational triage
effects of neighborhoods on education
effects of No Child Left Behind
effects of schools
effects of Teach for America
elite education
ETS
Everyday Antiracism
excessed teachers
exit exams
experienced teachers
Fordham and Ogbu
Fordham Foundation
Frederick Douglass High School
Gates Foundation
gender
gender and education
gender and math
gender and science and mathematics
gifted and talented
gifted and talented admissions
gifted and talented program
gifted and talented programs in New York City
girls and math
good schools
graduate student union
graduation rate
graduation rates
guns in Chicago
health benefits for teachers
High Achievers
high school
high school dropouts
high school exit exams
high school graduates
high school graduation rate
high-stakes testing
high-stakes tests and science
higher ed
higher education
highly effective teachers
Houston Independent School District
how to choose a school
IES
incentives in education
Institute for Education Sciences
is teaching a profession?
is the No Child Left Behind Act working
Jay Greene
Jim Liebman
Joel Klein
John Merrow
Jonah Rockoff
Kevin Carey
KIPP
KIPP and boys
KIPP and gender
Lake Woebegon
Lars Lefgren
leaving teaching
Leonard Sax
Liam Julian

Marcus Winters
math achievement for girls
McGraw-Hill
meaning of high school diploma
Mica Pollock
Michael Bloomberg
Michelle Rhee
Michelle Rhee teacher contract
Mike Bloomberg
Mike Klonsky
Mike Petrilli
narrowing the curriculum
National Center for Education Statistics Condition of Education
NCLB
neuroscience
new teachers
New York City
New York City bonuses for principals
New York City budget
New York City budget cuts
New York City Budget cuts
New York City Department of Education
New York City Department of Education Truth Squad
New York City ELA and Math Results 2008
New York City gifted and talented
New York City Progress Report
New York City Quality Review
New York City school budget cuts
New York City school closing
New York City schools
New York City small schools
New York City social promotion
New York City teacher experiment
New York City teacher salaries
New York City teacher tenure
New York City Test scores 2008
New York City value-added
New York State ELA and Math 2008
New York State ELA and Math Results 2008
New York State ELA and Math Scores 2008
New York State ELA Exam
New York state ELA test
New York State Test scores
No Child Left Behind
No Child Left Behind Act
passing rates
Pearson
picking a school
press office
principal bonuses
proficiency scores
push outs
pushouts
qualitative educational research
qualitative research in education
quitting teaching
race and education
racial segregation in schools
Randall Reback
Randi Weingarten
Randy Reback
recovering credits in high school
Rick Hess
Robert Balfanz
Robert Pondiscio
Roland Fryer
Russ Whitehurst
Sarah Reckhow
school budget cuts in New York City
school choice
school effects
school integration
single sex education
skoolboy
small schools
small schools in New York City
social justice teaching
Sol Stern
SREE
Stefanie DeLuca
stereotype threat
talented and gifted
talking about race
talking about race in schools
Teach for America
teacher effectiveness
teacher effects
teacher quailty
teacher quality
teacher tenure
teachers
teachers and obesity
Teachers College
teachers versus doctors
teaching as career
teaching for social justice
teaching profession
test score inflation
test scores
test scores in New York City
testing
testing and accountability
Texas accountability
TFA
The No Child Left Behind Act
The Persistence of Teacher-Induced Learning Gains
thinktanks in educational research
Thomas B. Fordham Foundation
Tom Kane
Tweed
University of Iowa
Urban Institute study of Teach for America
Urban Institute Teach for America
value-addded
value-added
value-added assessment
Washington
Wendy Kopp
women and graduate school science and engineering
women and science
women in math and science
Woodrow Wilson High School