Yesterday, I wrote about Morgaen Donaldson’s research on the survival rates of three cohorts of Teach for America teachers in their initial placement schools and in teaching overall. Today, I’ll describe one of her analyses of why TFA teachers leave their schools, focusing on the complexity of the teaching assignment and the corps member’s academic preparation for the subject(s) that she or he taught.

For this analysis, a complex teaching assignment for an elementary school teacher is one in which the teacher teaches more than one grade in a given year. Similarly, a complex teaching assignment for a secondary teacher is one in which the teacher was assigned to teach more than one subject in a given year. Many TFA recruits had complex teaching assignments during the years of observation. Between 16 and 20% of elementary TFA teachers were assigned to teach more than one grade in a given year, and 35% to 50% of secondary TFA teachers were assigned to teach more than one subject in a given year. (Note that this is different than teaching one grade in 2003 and a different grade in 2004, or one subject in 2004 and a different subject in 2005. These too might make teaching more complicated, but it’s across years rather than within them.)

In the 2000, 2001 and 2002 TFA cohorts, the vast majority of corps members majored in the social sciences or humanities in college. 52% were social science majors, 20% were English majors, 3% majored in the arts, and 6% majored in a foreign language. In contrast, about 5% were math majors, and 15% majored in science, computer science or engineering. Just 2% were education majors, and 4% majored in other subjects. (I think the numbers can exceed 100% due to double majors, but the text isn’t entirely clear on this.)

But their teaching assignments often differed dramatically from their formal academic preparation. One-half to three-quarters of the TFA recruits teaching secondary math were not math majors, and 38% to 50% of science teachers lacked a science major. Even in social studies, 16% to 31% of the TFA teachers were teaching out of their major field. Donaldson reports that out-of-field teaching diminished the longer a TFA teacher stayed in teaching.

Among elementary TFA teachers, a multiple-grade assignment increased the odds of leaving the initial school during or at the end of the first year of teaching by a factor of 3.29 (a probability of 19.1% for multiple-grade teachers, and 6.7% for single-grade teachers.) For the most part, this type of complexity did not influence retention in subsequent years, with the singular finding that in year 4, multi-grade teachers were significantly less likely to leave their initial schools than single-grade teachers. Many of the multi-grade teachers leaving their initial placement school in the first year transferred to another school, but multi-grade teachers also were more likely to leave teaching altogether in the first year than single-grade teachers.

At the secondary level, TFA recruits teaching multiple subjects were more likely than single-subject teachers to leave their initial placement schools and the field of teaching altogether in their first year of teaching. Beyond this first year, however, there were no significant differences in the likelihood of leaving the initial placement school, but multiple-subject teachers had a greater chance of leaving teaching altogether.

The out-of-field teaching story is complicated, with TFA math teachers teaching out of field more likely to leave their initial placement schools and the occupation of teaching altogether, and social studies teachers teaching out of field more likely to leave teaching. Oddly, science teachers lacking a science major were less likely to leave teaching than science teachers with a science major.

These patterns suggest that, at least in the years 2000, 2001 and 2002, TFA teachers often faced very complex teaching assignments for which they were not well-prepared academically, and the complexity of these assignments heightened the risk of leaving the initial placement school or of leaving teaching altogether. As I noted yesterday, there’s no comparison group, so we don’t know if novice teachers in these schools arriving via the traditional route had similarly complex assignments. Nor do we know if this pattern holds for more recent cohorts of TFA recruits, as there have been six cohorts since the three that Morgaen Donaldson studied.

One thing seems clear, however. If we want novice teachers to stay in their initial schools and to stay in teaching, they need adequate support as they learn their craft in the first years of teaching. Asking teachers to teach multiple grades, multiple subjects and/or subjects out of their college major fields is a peculiar way of supporting them.

A program note: We're going to take a break here for the next 10 days or so. eduwonkette and I wish you happy holidays!

Posted by skoolboy at 7:37 AM | Permalink | 12 Comments | Recommend

skoolboy remains fascinated by the way in which Teach for America, a program serving perhaps 3% of the students in the districts in which it operates, can seem like the tail wagging the dog. Like eduwonkette, I see many virtues to the program, but do not view it as a solution to the nation's challenge of developing a corps of skilled career teachers to serve our children and youth.

TFA recruits make a two-year commitment to teaching in a high-needs school, and the limited nature of this commitment is a recurring source of concern. If TFA recruits stay just two years and then leave, then the schools they serve face a revolving door of teachers shuffling in and out. TFA, for its part, cites recent evidence that TFA recruits are at least as effective in the classroom as other novice teachers. Moreover, TFA champions the enduring value of having its recruits see the challenges facing high-needs schools, if only for a few years, and claims that many recruits stay in the field of education beyond the two-year commitment.

There’s some new evidence on this latter point, emerging in the doctoral dissertation research of Morgaen Donaldson, formerly with Harvard’s Project on the Next Generation of Teachers, and now an Assistant Professor in the School of Education at the University of Connecticut. Donaldson surveyed the 2000, 2001 and 2002 cohorts of TFA recruits, obtaining 2029 responses, for a 62% response rate. Focusing on voluntary departures (approximately 16% in the sample were involuntary), she modeled the likelihood of staying in the initial placement school over time, as well as the likelihood of transferring to another school or leaving teaching altogether.

The charts below are from fitted hazard models that describe the cumulative probability of "survival" in the initial placement school across years, as well as the probability of voluntarily resigning from teaching for the first time. The first chart shows that about 90% of TFA recruits (voluntarily) remain in the initial placement school for a second year, and about 44% stay for a third year. These figures decline steadily over time, with about 22% staying in the initial placement school for a fourth year, 15% for a fifth year, and 9% for a sixth year.

The probability of voluntarily staying in the teaching profession over time is higher than the likelihood of staying in the initial placement school, since some TFA recruits, like teachers in general, transfer to other schools. The fitted models suggest that about 94% of TFA recruits remain in teaching for a second year, and 60% teach for a third year. 44% remain in teaching for a fourth year, 35% for a fifth year, and 29% for a sixth year.

It’s difficult to know whether to think of these rates of persisting in the initial school placement or in teaching at large are high or low. As usual, the question is, compared to what? TFA recruits are placed in schools that are claimed to be "hard to staff," and they may be challenging places to work, regardless of the route that brought the teachers to such schools. If the attrition rates for other novice teachers in these schools are just as high as those observed for TFA recruits, it’s harder to argue that TFA is exacerbating the problem of building a stable, high-quality teaching force in high-needs schools. Donaldson’s study doesn’t shed any light on this issue.

I’ll have a bit more to say about Morgaen Donaldson’s research on how working conditions affect the persistence of TFA recruits in their initial schools tomorrow.

Posted by skoolboy at 6:29 AM | Permalink | 17 Comments | Recommend

skoolboy still has nothing substantive to say about Arne Duncan. But he's pleased to note that Duncan's a member of the tribe: his B.A. from Harvard is in sociology. Duncan took a year off from school to write a senior thesis on life in Kenwood, the south side Chicago neighborhood in which his mother Sue had founded an after-school program in 1961. Duncan's 123-page thesis, entitled "The values, aspirations and opportunities of the urban underclass," was read and praised by William Julius Wilson, among the most eminent urban sociologists of our time.

Duncan's appointment will vault him into the list of prominent Americans who majored in sociology. It's not a long list! Ronald Reagan double-majored in economics and sociology, and Rev. Martin Luther King, Jr. was a sociology major. So too Rev. Jesse Jackson, novelist Saul Bellow, and a number of other prominent civil rights leaders and members of Congress. And Michelle Obama majored in sociology at Princeton.

But before too long, we're into B-list celebrities and leaders: Dr. Ruth, Regis Philbin, a smattering of NFL and NBA stars, Robin Williams, and a couple of Canadians: Dan Aykroyd and Late Show bandleader Paul Shaffer.

Oh well. At least there aren't any well-known crooked sociologists. Rod Blagojevich was a history major at Northwestern, and as for Bernie Madoff? Beats me. Nobody goes into sociology for the money.

Posted by skoolboy at 11:53 AM | Permalink | 2 Comments | Recommend

skoolboy has absolutely nothing of substance to say about Education Secretary nominee Arne Duncan, whom he has met exactly once. But he continues to mouth off about New York City's Teacher Data Reports, the NYC Department of Education's version of value-added assessment. Which are not to be used to evaluate teacher performance. But rather for instructional improvement. Excuse me, skoolboy has something in his eye.

It's hard not to view these Teacher Data Reports as a Trojan Horse. Just how is a tool that is designed for capacity-sorting supposed to function for capacity-building? After all, a teacher value-added measure might tell us something useful about which teachers are more or less successful in raising their students' test scores, but it tells us nothing about the specific instructional practices that account for their relative success.

How are Teacher Data Reports supposed to improve instruction? In her videotaped comments to teachers, Amy McIntosh, the Chief Talent Officer at NYC's Department of Education, says, "These reports will provide information that will help teachers and school leaders gain insights about important aspects of a teacher's practice ... Whether individual teachers have a greater influence on the learning of some groups of students than on others ... Finally, we can see what teachers might benefit from development focused on, say, the needs of English language learners, and which teachers might be best positioned to lead that kind of professional development ... We also think they will ... help you think about how you can share the techniques you use with your colleagues in your school or across the city."

Hmm. So the specific strategies for improving teaching practice are what, exactly? Having more successful teachers lead the professional development of less successful teachers? Expert practitioners don't always make expert coaches. Hall-of-Fame pro basketball player Isiah Thomas--unquestioned as one of the best point guards of all time--was a mediocre coach for the Indiana Pacers and New York Knicks.

Here's why. Teaching is an extraordinarily complex activity, with teachers making thousands of decisions in the course of their work. Successful teachers make many good decisions and some bad decisions, whereas less successful teachers make many bad decisions and some good decisions. But the capacity to reflect on one's practice and figure out which of those decisions are good and which are bad is exceedingly rare, as is the capacity to share this knowledge with others. In the absence of this reflective capacity, we're all prone to attribute our successes and failures to our pet theories, which may or may not be correct. A Teacher Data Report that provides reassurance that a teacher is successful will only solidify and reinforce a personal folk theory about the reasons for that success.

Yet the Teacher Data Report provides no evidence whatsoever about why a teacher is successful--the many daily practices that promote student learning. And if a teacher's personal theory is inaccurate, then sharing it with others will not improve instruction, nor student achievement. It could even make things worse, focusing attention on ineffective practices. A tool like the Teacher Data Report that claims to be useful for increasing teachers' capacity to teach students effectively, but instead is only useful for ranking teachers on their effectiveness, is a modern-day Trojan Horse.

Posted by skoolboy at 7:50 AM | Permalink | 5 Comments | Recommend

This is too precious not to comment on: the Center for Education Reform, the organization that sponsored that full page ad slamming the AFT charter study and the Times in 2004, threw this celebratory paragraph into their newsblast today (see background here):

WHAT'S WORKING. D.C. charter schools are succeeding, according to The Washington Post, and "have opened a solid academic lead over those in [the city's] traditional public schools." An analysis of test results for economically disadvantaged students shows that "D.C. middle-school charters scored 19 points higher than the regular public students in reading and 20 points higher in math." D.C. charter school students outscored their conventional public school counterparts in other areas as well. In an online chat, writers discussed some of the reasons charters are thriving in the District: "A culture where the grown-ups trust each makes it a lot easier to teach kids. The experts doing research say high expectations and standards of behavior have to be applied consistently across classrooms. That's a lot more evident at charters than most DCPS schools I've visited. Charter directors were very consistent in saying that they will not hesitate to get rid of teachers who they feel are not performing. On the other hand, they are also very eager to keep the teachers they like, and provide support and encouragement and training to keep them happy."

This is a far cry from the fire-breathing condemnation of that ad, isn't it?: "The study in question does not meet current professional research standards. As a result, it tells us nothing about whether charter schools are succeeding."

You can't make this stuff up.

Posted by at 9:34 PM | Permalink | 17 Comments | Recommend

Recall the Great Charter School War of 2004: After the NY Times published the results of an AFT report finding that traditional public schools outperformed charters, all hell broke loose. Every charter school advocate and their mother intervened in the name of educational research, arguing that the study was fundamentally flawed and that the Times story was biased against charter schools. Shortly thereafter, charter advocates took out a full page ad in the Times blasting the study and the Times for putting it forward.

To be sure, students are not randomly assigned to charter schools, so these critiques were not without merit. So keep your eyes on the Washington Post over the next week and see if charter school advocates again swoop in to defend educational research from the bad guys who would misuse it for their own purposes. Yesterday, the Post published an almost identical analysis claiming that DC charters, which currently enroll a third of DC students, "have opened a solid academic lead over those in its traditional public schools."

Again, for those who are willing to endure the occasional recycled rant on selection versus treatment effects:

First, that students selected into a charter lottery makes them different from those who did not. It may be that their parents are more involved in their education, that they are having a particularly bad experience at their neighborhood school, or that their parents can no longer pay for private school. Whatever the reason, families selecting in, even if they are all poor and minority kids, are different by virtue of choosing a non-neighborhood school.

Lots of choice advocates will spar on this point, and argue that everyone wants a better choice for their children, so there is no selection problem. While rhetorically effective, anyone arguing that families that choose into a charter school are the same as those who don’t is simply wrong. Saying that two-thirds of the kids are poor and that the overwhelming majority are African-American and Hispanic doesn't solve this problem. (Btw, in the small schools context, this is Joel Klein's favorite pasttime.) Even if charter kids had prior test scores identical to their neighborhood school peers, we still couldn't compare charter and neighborhood school kids who didn't opt in with any confidence because there is selection on unobservables - things like motivation and aspirations that are not measured by administrative datasets used to make these comparisons.

The only defensible approach here is to compare students who entered the charter lottery and won with those who entered the lottery and lost. But if the Post wants to extend its logic, here's a free story idea from your friendly neighborhood Secret Santa: go ahead and compare the NAEP results of charters and public schools nationally (and for extra credit, go one step further and compare the outcomes of free lunch kids in charter and traditional public schools), and let's get the Charter War of 2008 started.

Posted by at 10:56 PM | Permalink | 31 Comments | Recommend

"Don’t think about elephants," skoolboy’s father used to joke, long before George Lakoff’s manifesto with a similar name. The joke, of course, is that by trying not to think about elephants, all that you can think about is elephants. The harder I tried not to think about elephants, the more I thought about them.

The New York City Department of Education has its own variation. This month, the DOE is sending Teacher Data Reports, which purport to estimate the effect of individual teachers in grades 4-8 on students’ test scores, to school principals, who will then distribute the reports to their teachers after the principals have been trained. "The Teacher Data Reports are not to be used for evaluation purposes," wrote Chancellor Joel Klein and UFT President Randi Weingarten in an October letter to teachers. "That is, they won’t be used in tenure determinations or the annual rating process. Administrators will be specifically directed accordingly." Similarly, the Frequently Asked Questions section of the DOE’s Teacher Data Tool Kit website poses the question "How can you be sure that principals won’t use the Teacher Data Reports to evaluate teachers?" The response: "Principals have been and will continue to be explicitly instructed not to use Teacher Data Reports to evaluate their teachers. The DOE has standard processes in schools for teachers to raise issues or concerns."

And yet. From the Frequently Asked Questions on the DOE’s Teacher Data Toolkit website: "By isolating individual teachers’ contributions to student progress, the Teacher Data Reports provide valuable information to school leaders and teachers about where to focus instructional improvement efforts. …Teacher Data Reports provide information about how individual teachers’ efforts influence student learning … A sophisticated multivariate regression analysis based on NYC data from 1999-2008 determined how much to weigh each factor [to calculate students’ predicted gains] … A panel of technical experts has approved the DOE’s value-added methodology. The DOE’s model has met recognized standards for demonstrating validity and reliability. Teachers’ value-added scores from the model are positively correlated with both School Progress Report scores and principals’ perceptions of teachers’ effectiveness, as measured by a research study conducted during the pilot of this initiative."

In other words: The Teacher Data Reports rely on sophisticated statistical techniques that are valid, reliable and approved by experts, and they isolate an individual teacher’s contributions to student learning. But, you principals who are under tremendous pressure to increase test scores or face losing your jobs, don’t you dare think about using these Teacher Data Reports to evaluate teachers.

Don’t think about elephants.

Posted by skoolboy at 7:18 AM | Permalink | 8 Comments | Recommend

A thought experiment: If Malcolm Gladwell, author of Outliers, was to jello wrestle his alter ego on central matters of public education, who would come out on top?

In his article in the New Yorker this week, Gladwell's argument is that it's hard to predict who will become a great pro quarterback or teacher before job candidates start playing or teaching. Like most engaged in the teacher quality debate, Gladwell assumes that there are "good" and "bad" teachers, and this quantity exists a priori. But it's just impossible to observe it before a teacher steps into the classroom. It's not about training. And it's not about some schools providing more supportive environments for teaching than others. For Gladwell, it's about individual "withitness," and we can't see it until after the teacher has walked through her classroom door.

It was surprising to see Gladwell focus so heavily on the potential of the individual player or teacher, given that he just penned a book about the importance of social contexts and chance in producing human greatness. As he put it, "The tallest oak in the forest is the tallest not just because it grew from the hardiest acorn; it is the tallest also because no other trees blocked its sunlight, the soil around it was deep and rich, no rabbit chewed through its bark as a sapling, and no lumberjack cut it down before it matured."

So where's the "forest" for a quarterback or teacher? It's a team. Or a school. Even the most gifted quarterbacks end up with pretty crappy pass completion stats if their teammates consistently miss the ball. And a great quarterback doesn't look so great if he's a poor fit for the team he's playing with. The same goes for teachers. So my fingers are crossed that the Gladwell who recognizes the importance of the environments - not just individuals - wins this match.

Posted by at 12:01 AM | Permalink | 24 Comments | Recommend

Hilary Levey is a doctoral candidate in Sociology at Princeton University. Below, she shares findings from her dissertation, "Playing to Win: Childhood, Competition, and the Credentials Bottleneck."

Many parents work more hours outside of the home and their lives are crowded with more obligations than ever before; many children spend their evenings and weekends trying out for all-star teams, travelling to tournaments, and eating dinner in the car. What explains the increase in children’s participation in activities outside of the home, structured and monitored by their parents, when family time is so scarce? As the parental “second shift” continues to grow, alongside it a second shift for children has emerged — especially amongst the middle- and upper-middle classes — which is suffused with competition rather than mere participation. What motivates these particular parents to get their children involved in competitive activities?

Using evidence from three case studies (one academic, chess; one artistic, dance; and one athletic, soccer) drawn from 16 months of fieldwork, and 172 interviews — with 95 parents of elementary school-children, 37 of those elementary school-age children, and 40 teachers and coaches — I argue in my dissertation, Playing to Win, that the extensive time devoted to competition is driven by parents’ demand for credentials for their children, which they see as a necessary and often sufficient condition for entry into the upper-middle class and the good life that accompanies it. At the same time, of course, this new form of early competition reinforces a “less than level” playing field among children, in terms of class and gender.

That American families are busy is not surprising, especially to those who study family life and those who live it. But it’s not just that middle class kids spend their time in organized activities. What is critical, and rarely discussed, is the competitive nature of their extra-curricular lives. Many activities that were previously non-competitive have been transformed from environments that only emphasized learning skills, personal growth, and simple fun to competitive cauldrons in which only a few succeed.

Such competitive experiences were once limited to high school. Students entered athletic contests, joined debate teams, built “careers” as high school newspaper editors, and in hundreds of other ways sought to distinguish themselves in adolescence. For millions of middle class American children today, waiting until high school to prove one’s mettle would be a big mistake. The bottlenecks these kids worry about and will face require much more advanced preparation. Even the preschool set is busily trying to stand out from the crowd!

It is tempting to denounce these preoccupations as the hyper-fixation of neurotic parents who are living through their children, and many pundits are not shy about invoking analyses that are just shy of pathology. These parents are labeled helicopter parents who hover over their kids from infancy through college graduation, even until children secure employment after college. But are these parents crazy? No. Their children face very real bottlenecks through which they need to pass if they are going to achieve in ways similar to their parents. And the probability of that outcome appears to their parents — with good reason — to be less than it once was.

At the same time it would be a mistake to think that parents of kids as young as seven fixate on college admissions offices every Saturday out on the soccer field. Instead, they understand the grooming of their child as producing a certain kind of character and a track record of success in the more proximate tournaments of sports or dance or chess. (But were parents to think in directly instrumental terms about a thick admissions envelope, they would not be far off the mark: activity participation, particularly athletics, does confer admissions advantages, either through athletic scholarships or an admissions “boost,” giving students an edge when applying to elite schools.) These competitive activities are seen by many parents as the essential proving ground that will clear their children’s paths to the Ivy League because they help children acquire skills and focus their time and energy.

Playing to Win illustrates the ways in which competition is now a central aspect of American childhood for many, showing that countless boys and girls no longer simply play — they play to win.

Posted by at 10:07 PM | Permalink | 22 Comments | Recommend

I promise that this whole week won't be deadly depressing, but Alexander Russo threw down the gauntlet about the media's lack of attention to results in Chicago Public Schools under Arne Duncan. So I took a look at Chicago's NAEP performance.

Have gaps separating white/black and white/Hispanic students in Chicago shrunk in the last 5-6 years?

Nah.

There are no statistically significant declines in these gaps in 4th or 8th grade reading or math. In many cases - for example, 4th and 8th grade math and 8th grade reading - it's not that the black-white achievement gap is declining, but not by enough to be statistically significant. These gaps are actually growing. Sigh.

Lucky for Duncan, Joel Klein doesn't have a leg up in this area - neither superintendent has been successful in narrowing racial achievement gaps. (See NYC achievement gap analyses in painful detail here.)

And while we're on the topic of Joel Klein and NAEP scores, a word on his latest statistical gaffe about sampling. From the outback, here's the exchange between a reporter and Klein at his Australian National Press Club talk:

Reporter: Are you concerned that the only independent national assessment of American children shows that there is actually no improvement in New York, with the slight exception of year four maths?

Klein: [The NAEP tests] are sampled, whereas our state tests are mandatory. They're the accountability and, if you look at the data that I provided to you, across the board those state tests show that we're outperforming the rest of the state and everyone else.

Given that a sample is appropriately drawn - and there is no reason to believe that the NYC NAEP sample is not - it should produce unbiased estimates of the achievement of the underlying population, NYC kids. This guy is being discussed as a candidate for Secretary of Education, and in the same breath telling Australians that our national test is inaccurate because it's based on a sample. Oy.

Posted by at 9:52 PM | Permalink | 14 Comments | Recommend

We hear a lot about "gap closing" schools these days, though this term often gets tossed around loosely. Consider Steven Wilson's recent report on "gap closing" Boston charter schools, in which gap closing schools are defined as, "schools that serve students of color from economically disadvantaged families and post achievement levels that rival - and sometimes exceed - suburban school districts."

Gap closing, according to Wilson, refers to proficiency rates on state tests, and herein lies the rub. It's possible for gaps to appear to be closing on state tests if we rely on proficiency levels, even as wide gaps persist when we consider average state tests scores or other measures of performance. And by thrusting these schools onto center stage and making the claim that schools - right now - are entirely eliminating disadvantage, we end up perverting the debate about what it takes to reduce inequality.

Wilson profiles three gap closing high schools - Academy of the Pacific Rim, Boston Collegiate, and the MATCH school - and reports that these schools are exceeding state passing rates on the MCAS test. Hence, they are "gap closers." Surely these schools deserve props, but we continue to see quite substantial gaps between their students and the rest of the state when we look at these schools' SAT scores. (Normal caveats on SAT scores here: Everyone does not take the SAT, but the latent claim in the "gap closing" debate is that students at these high-flying schools walk out as "college ready" as college going students from more advantaged families.)

Below, I list the combined 2007 reading/math score for these schools and a few comparison groups (See Massachusetts SAT scores and school scores here.):

* White Massachusetts students: 1061
* Massachusetts state average: 1035
* MATCH (23 test takers): 988
* Boston Collegiate (14 test takers): 964
* Academy of the Pacific Rim (34 test takers): 932

In short, we've still got a gap - kids at these gap closing schools are lagging behind their more advantaged college-going peers. The two take home points here? Beware of gap claims made based on proficiency rates, and don't count on schools - even great ones - to remedy the substantial disadvantages that have accrued over these students' lifetimes.

Image Credit: E3

Posted by at 11:17 AM | Permalink | 5 Comments | Recommend

The recent flurry of attention to high school completion rates has revived interest in early warning systems designed to identify students at risk of dropping out of high school. The idea behind these early warning systems is that, through the analysis of administrative data, schools and school districts can develop models of risk factors which predict a high probability of dropping out of high school. If the models successfully distinguish probable dropouts from probable graduates, students at high risk of dropping out can be identified, and support resources can be focused on these students identified as at risk of dropout.

A good early warning system will have high sensitivity and high specificity. High sensitivity means that the early warning indicators will identify a very high percentage of those youth who will eventually drop out (i.e., a high percentage of "true positives"). High specificity means that the indicators will not identify many youth who are not destined to drop out (i.e., a low percentage of "false positives".) Phil Gleason and Mark Dynarski of Mathematica Policy Research showed in the federally-funded School Dropout Demonstration Assistance Program evaluation that most dropout prevention programs had disappointingly low sensitivity and specificity: they failed to serve youth who would eventually would drop out, and they frequently served youth who would likely have graduated in the absence of the program.

Early warning indicators have been developed in Chicago, by Elaine Allensworth and John Easton, and in Philadelphia, by Robert Balfanz and Ruth Curran Neild, as well as other cities. The Chicago indicator is an indicator of being "on-track" for high school graduation; a student is "on track" if he or she earns at least five full-year course credits and no more than one F in one semester in a core course during the first full year of high school. The Philadelphia measure relies on sixth-grade measures of academic performance and behavior. A student with at least one of the following four characteristics had at least a 75% chance of dropping out of high school: (a) a final grade of F in math; a final grade of F in English; attendance below 80% for the year; and a final behavior mark of "unsatisfactory" in at least one class.

It’s not exactly rocket science to show that students who fail courses and have low attendance have an elevated risk of dropping out of high school, but the architects of these systems argue that the specific indicators that students manifest warrant different responses. Low attendance may stem from a different set of sources than poor behavior, for example, and a key feature of these indicator systems is that they frequently rely on administrative representations of students’ behavior in the school and classroom (especially the incidence of failing a core course), rather than more distal status measures that are less amenable to a programmatic response. Finding that low-SES youth are more likely to drop out, for example, would not give a school or district much to work with.

One issue to consider is the way in which early warning indicators are used in medicine. They’ve become controversial in instances in which the indicators don’t prescribe a reliably successful course of treatment. In the absence of an effective treatment plan, critics argue, indicators of the heightened risk of conditions such as prostate cancer or breast cancer may simply upset patients and not improve outcomes. In contrast, cholesterol tests are much more valuable as early warning indicators for heart disease because the use of statins to reduce cholesterol levels is recognized as an effective treatment that improves cardiovascular outcomes.

The question we might ask about dropout prevention is: If we knew that particular students had an elevated risk of dropping out of high school, what would we do differently? The problem here is that we do not have a dropout prevention wonder drug that has shown to be reliable in lowering dropout rates in multiple contexts. The history of dropout prevention research is littered with poorly-designed, small-scale research studies that have failed to identify a set of program elements that consistently work. Moreover, the best-designed of such studies have found modest program effects on the probability of dropping out.

None of this is to say that local efforts to reduce dropping out are ineffective. Many talented and motivated people lead and staff such programs, and they may in fact reduce the risk of dropping out for some groups of youth. The problem is that we don’t really know if they work or not. And in the absence of such knowledge, skoolboy is just not sure that early warning systems to identify potential dropouts are all that useful.

Posted by skoolboy at 9:56 AM | Permalink | 18 Comments | Recommend

A month ago, Flypaper asked us to come up with appropriately silly backdrops for Margaret Spellings' portrait, which will be unveiled on December 18th.

All you lame duck Department of Education staffers - here's something to post on the water cooler tomorrow morning. Enjoy.

Posted by at 3:31 PM | Permalink | 4 Comments | Recommend

Sean Corcoran is an economist who teaches at the Steinhardt School of Culture, Education, and Human Development at NYU.

One of my all-time favorite bumper stickers is the now-classic:

To my knowledge, the Air Force has yet to experiment with bake sales. But—according to three papers presented at last month’s National Tax Association meeting in Philadelphia—private contributions through local education foundations have become a significant source of operating funds for many of the country’s public schools.

Education foundations are not your grandmother’s PTA. School foundations organize as 501(c)(3) corporations, and in some cases mount sophisticated fund-raising campaigns, soliciting contributions from local businesses, parents, and philanthropies. Foundations fund more than occasional trips to the zoo, paying to supplement instructional programs, offer scholarships, and provide extra pay to recruit and retain teachers.

School foundations have grown over time. According to a 2005 report by Eric Brunner and Jennifer Imazeki, contributions to California school foundations rose from $123 million in 1992 to $238 million in 2001. If these contributions were divided up evenly statewide, they would amount to only $40 per child. Of course—as Brunner and Imazeki point out—these contributions are far from evenly distributed. Donations are strongly related to family income, and in some cases they are quite high, at more than $250 to $500 per student. (You can read about the $3.3 million education foundation in Santa Monica-Malibu Unified School District here).

What explains the growth in private foundations? Two papers presented at the NTA meeting offer some hints. Julie Golebiewski of Syracuse University linked foundation giving in California to the restrictiveness of tax limitations in that state (the famous Proposition 13). In a nutshell, she finds that school districts that would have spent more on schools in the absence of the limitation were much more likely to raise funds through private foundations. Similarly, Tom Downes of Tufts found that private contributions in Vermont were highest in wealthy districts who—under their 1997 finance reform—would have been penalized for taxing themselves at higher rates.

Why do these education foundations matter? I find them interesting for several reasons. First, they stand in sharp contrast to the usual claim that school spending is out of control, and that teachers are adequately paid. If this were true, how can we account for the growth in voluntary contributions to public schools (some of which is channeled to teacher compensation)?

Second, foundations have the potential to undo school finance formulas designed to equalize educational opportunities. The evidence on school foundations suggests that communities with a high demand for school quality (or relative school quality) will find a way to meet this demand, regardless of the rules put before them. Thus, a policy based on promoting equity will not necessarily result in greater equity.

Finally, school foundations are likely to grow in importance as public education continues to decentralize control to individual schools. Charter schools and schools funded through a “weighted student” formula may come to rely on private giving whenever public funds are insufficient to meet the unique demands of their constituencies. As a result, school policies designed to level the playing field may end up tipping the balance in favor of schools most able to mobilize private resources. (For a terrific example from New York City, see this chapter by one of my colleagues Amy Schwartz).

Whether or not local school foundations will play a major role in the future of school funding remains to be seen. Eduwonkette readers, what do you think? Do private contributions play a role in your school district? Should they?

Posted by at 12:30 PM | Permalink | 36 Comments | Recommend

DC Schools Chancellor Michelle Rhee is on the cover of this week's Time magazine. The accompanying article features a striking statistic: according to her office, she answered 95,000 e-mails last year. Allow skoolboy to speculate about this figure.

Let's suppose that Chancellor Rhee responds to e-mail seven days a week, and that she worked 50 weeks last year. (skoolboy would hope that she worked less, because that's a grueling pace.) 95,000/350 is about 270 e-mails per day to which she responded. Suppose further that it takes one minute to read and respond to an e-mail. (Some will take more; few, I imagine, could take less.) That's a minimum of 270 minutes per day, or 4 1/2 hours per day of e-mail. Every day. Seven days a week, 50 weeks a year.

Amanda Ripley, the author of the article, describes spending a day with the Chancellor in August as she made unscheduled visits to DC public schools:

She emerged from her chauffeured black SUV with two BlackBerrys and a cell phone and began walking--fast--toward the front door of the first school... When we got inside, she walked into the first classroom she could find and stood to the side, frowning like a specter. When a teacher stopped lecturing to greet her, she motioned for the teacher to continue. Rhee smiled only when students smiled at her first. Within two minutes, she had seen enough, and she stalked out to the next classroom.

Later, Ripley writes, "She reads her BlackBerry when people talk to her. I have seen her walk out of small meetings held for her benefit without a word of explanation. She says things most superintendents would not. 'The thing that kills me about education is that it's so touchy-feely,' she tells me one afternoon in her office."

skoolboy finds all of this fascinating, and appalling. He's seen parallels in New York, with everyone from the Chancellor on down furiously thumbing their BlackBerries in meetings with real, live people who are trying to talk to them about issues they care about. Has technology fundamentally transformed the nature of leadership in educational organizations, reducing the need for sustained engagement with interested stakeholders around social, cultural and political issues? Can a big-city school superintendent really manage by e-mail?

There's always a danger of overinterpreting a journalistic account, and more data on the linkage between technology and theories of school leadership would provide valuable context. In the meantime, when it comes to Chancellor Rhee and her peers' preference for BlackBerries to people, maybe the medium is the message.

Posted by skoolboy at 8:30 AM | Permalink | 24 Comments | Recommend