A new report from Brookings’ Russ Whitehurst and Michelle Croft concludes that high-quality charter schools--not community, social and family services--are responsible for the success of Harlem Children’s Zone (HCZ) in improving student achievement. And, based on this finding, the report argues for skepticism about the potential of Broader, Bolder type efforts to improve student achievement through social and community services, as well as President Obama’s Promise Neighborhoods program to replicate HCZ-like initiatives elsewhere.
There’s an important point here: HCZ has gained national attention, plugs from President Obama and Secretary Duncan, and praise from across the political spectrum--sometimes it seems like the only thing everyone in the fractious education policy community can agree on!--in large part because its model of quality schools combined with a network of social services for children and families offers something everyone can love. But I’ve always found it incredibly frustrating that the very element that makes HCZ so popular and compelling--its integration of social and community services--is also its least well-understood and evaluated component. Paul Tough’s widely read book on HCZ spent most of its time focused on the Zone’s Promise Academy charter schools, with chapters on its pre-k and “Baby College” parent education programs, but said almost nothing about the after-school, asthma prevention, violence prevention, and other social programs that HCZ offers. Proponents of a more social-services oriented approach to education reform hold up Roland Fryer’s and WIll Dobbie’s evaluation of Harlem Children’s Zone’s charter schools as evidence that their approach works--but Fryer and Dobbie in fact couldn’t find any evidence that social and community services contributed to the charter schools’ strong results (which included closing the black-white achievement gap for middle school students in math).
Whitehurst and Croft seek to build on Fryer and Dobbie’s findings by comparing the performance of HCZ charter schools to other Manhattan and Bronx charters that do not offer similar social service supports--and they find that HCZ ranks right in the middle of the pack of New York’s (admittedly, disproportionately high-performing) charter schools, with a number of schools that don’t provide HCZ’s mix of social services coming out ahead.
That’s something worth knowing, although Jay Mathews raises some important objections and caveats here. And I’d note that while most high-performing charter schools do not explicitly say they provide the types of social supports HCZ says it does for kids, in many cases the “No Excuses” approach involves meeting kids and families where they are in ways that address social and family needs.
But I’m more curious about what, if any, HCZ’s social and community services have on children who live in the zone but do not attend the Promise Academy charter schools. This is an important question for two reasons: First, proponents of a more social-services based approach sometimes seem to be arguing for social and community services as a substitute for school reforms, rather than a complement to them--so it’s important to know what the relative impacts and cost-effectiveness of school-based reforms and social/community services interventions are. Second, there are a lot more kids in the Harlem Children’s Zone service area (about 8,000) than there are enrolled in the schools (about 1,200), so it’s important to know if HCZ is making a difference for those kids.
Unfortunately, getting answers here is tricky. As Dobbie and Fryer note, most of the social and family services in HCZ haven’t tracked the kind of data that would enable them to evaluate the impacts of these programs. But for those where Dobbie and Fryer did have data to assess their impact, the results weren’t promising. Participation in Harlem Gems, HCZ’s high-quality pre-k programs, is correlated with higher achievement in math and English language arts in third grade--not too surprising given evidence that already exists on the benefits of quality pre-k, but still an affirmation of the quality and effectiveness of HCZ’s pre-k program. But children’s and families’ participation in Baby College, HCZ’s signature parent education and support program, did not seem to have an impact on school readiness at age 4 or third grade test scores. As far as other social services are concerned, Dobbie and Fryer note, “there is substantial anecdotal evidence that Harlem Children’s Zone was unsuccessful in the years before opening the charter schools.”
Now, I don’t believe that any of this is necessarily an argument against seeking to improve social services for children and better integrate them with schools and education reform. For one thing, not every positive impact on children’s lives is reflected in student achievement data. Non-cognitive skills, mental health, and health can all influence individuals’ life trajectories and outcomes in ways that test scores don’t reflect (see James Heckman for more on this). It’s worth remembering that the landmark Perry Preschool study also didn’t really improve elementary test scores--but participants still did much better as adults. Baby College, Harlem Peacemakers, and other HCZ programs may be improving children’s non-cognitive skills, mental health, and well-being in other ways that will yield long-term gains. Similarly, programs that prevent child abuse or removal of children from their families can yield child benefits and taxpayer savings even if they don’t affect test scores. Moreover, there’s just a moral case for ensuring that children are cared for and their basic health, nutrition, nurturing needs met whether or not that “pays off” in test scores or taxpayer savings.
What I do believe the body of evidence from HCZ illustrates is that when it comes to improving children’s lives, social and community services are not a substitute for reforms that improve the quality of instruction and curriculum going on in their schools. To the extent that “whole child” reformers argue that addressing social and health needs will obviate the need for school reform, or that we can’t undertake school reforms until all children’s social service needs are met, they’re leading us down a blind alley.