City Year Schools Twice as Likely to See Math, English Boosts, Study Finds
Pleasant View School was one of a slew of high poverty schools in Providence, R.I., marked for an overhaul in 2012, but three years later, it is not only out of academic crisis, but thriving.
Pleasant View Principal Gara B. Field credits a big part of the school's revival to a team of young adult AmeriCorps members who have adopted the school as part of the City Year program's "Whole School, Whole Child" school wide initiative. "It's been a huge partner," she said. "One of the best things we ever did was write them into our school improvement plan."
Some new evidence released this morning suggests Fields isn't alone: Schools that participated in City Year's 150 school wide programs in 22 cities were more likely to see overall improvements on their states' mathematics and English/language arts tests than similar schools that did not participate, according to a new evaluation of schools in the nonprofit program's 150 schools.
City Year is a national nonprofit supported in part by the federal Corporation for National and Community Service that hires young-adult AmeriCorps to spend a year working and running programs in high-poverty schools. The group's school wide program uses teams of seven to 18 corps members who support teachers, as well as provide reading and math tutoring, attendance and behavior coaching, social-emotional activities, and after school homework help, and enrichment. Corps members serve in more than a quarter of schools eligible for federal school improvement grants.
The study, by the Washington-based research firm Policy Studies Associates, Inc., compared state test performance of schools which received services from City Year in a whole-school program, with local comparison schools matched on demographics and other factors. The researchers used surveys and administrative and testing data to track the performance of students overall in grades 3 through 8 and high school in math and language arts, as well as high school graduation, in 150 City Year schools and nearly 500 matched comparison schools. They also tracked how many students in the City Year schools were identified for "focus" support in math, reading, or social- emotional or behavior issues.
School practices—data reviews, shared teacher-planning time, appreciation and reward activities, community-partnership development, tutoring frequency, and progress monitoring of students—were used to evaluate the schools' levels of implementation.
Schools working with the program were about twice as likely as comparison schools to show overall improvements in language arts in each of the three study years: 2011-12, 2012-13, and 2013-14. In mathematics, City Year schools were also significantly more likely to show improvement than comparison schools in two of the three study years:
Because the schools were in different states, using different tests, the researchers calculated improvement simply by determining whether the percentage of students proficient on the state tests was higher in one year in than the previous year.
"What was surprising was the findings were consistent across all the sites," said Leslie M. Anderson, report co-author and the managing director of Policy Studies Associates, Inc. "They're working with a lot of schools, and there's clearly variation in implementation, but the results are certainly promising and worthy of more study. It suggests there's probably something going on in these City Year schools."
Schools with a higher number of corps members per 100 students were the most likely to show improvement, as were schools with a higher number of students on "focus" lists of more intensive intervention in math or reading.
Pleasant View, which serves preschool through grade 5, was one such school. Principal Field said in 2012 only 17 percent of the school's 480 students were proficient on state tests in math, a focus area for the corps members. Since then, "We've grown 10 percent every year [in math scores] and now we're performing above the district level and almost at the state level."
Anderson and co-author Julie Meredith, a research associate for Policy Studies Associates, examined individual school-level outcomes using a portion of the schools, but found significant effects only at the middle school level. Meredith and Anderson said the samples of elementary and high schools studied were too small to prove an effect from individual City Year interventions at those grades, but benefits seemed to be similar at different levels.
The researchers acknowledged several limitations to the study—for example, there was no base-year data collected on the schools before they began participating in City Year, so it is impossible to say whether they had started improving before the program began—but the consistency of the effects across all of the sites in different states was promising. "Had we found that the results varied significantly by subject, by school level, or by site, it would be difficult to draw any clear conclusions about the City Year model and its impact on schools," Anderson said in an email. "In fact, because we saw clear and consistently positive differences in CY schools versus their matched comparisons, by subject, by school level, and by site, we believe that these results support the conclusion that City Year is on to something with its Whole School Whole Child school improvement model and that further study is warranted."
Investing in Innovation
Educators won't have long to wait. City Year is also part of a nearly complete randomized control trial through the federal Investing in Innovation program. Research firm the MDRC research organization tracked students in 60 schools in 11 districts throughout every region of the country, comparing separate cohorts of students in City Year and matched comparison schools from grades 6 through 12 and grades 9 through 12.
That randomized study will look at differences in student absences, course performance, high school graduation rates, and other factors, and it will be accompanied by a separate implementation study digging into differences in school climate and other factors that may contribute to the students' progress. Both of the i3 studies are expected in December or January.
In the meantime, City Year President Jim Balfanz said the current study is helping the group focus its work. "It was interesting and promising for us to see," Balfanz said. "Here's a pretty consistent and positive suggestion around students' achievement. ... We're seeing the implementation schools have higher degrees of important practices [such as] teacher collaboration and support from instructional coaches. Our hypothesis is [the corps members] are freeing up teacher time to differentiate instruction."
For example, Rhode Island schools participating in City Year in 2012-13 were 25 percent more likely to improve in language arts during the study and 11 percent more likely to improve in math.
In Pleasant View, one City Year member served in each classroom, tutoring small groups and supporting students during computer work, Field said. The corps members also helped develop and put on eight "Homework Diners" catered by local restaurants, in which parents and nearly all of the schools' teachers come to discuss how parents can help their children with math assignments and play math-related games at home.
They have also helped mentor students with chronic absenteeism and other behavior problems, and Field said the school's suspension rate has dropped from 42 in 2012 to one so far this school year.
"Their impact is so deep and the connections so meaningful, you can't really measure it with numbers; it's the relationship piece that's wonderful," Field said. "Pleasant View wouldn't be Pleasant View without City Year."
You can read the full study here.
Photo: City Year AmeriCorp member Maria Diaz works with middle school students in Providence, R.I./Elliot Haney City Year.
Chart: A new evaluation of City Year finds participating schools in 22 cities were significantly more likely to improve in math and reading state tests than were comparison schools. Source: "Analysis of the Impacts of City Year's Whole School Whole Child Model of Partner Schools' Performance," Policy Studies Associates.