Nontraditional After-School Program Developing Metric to Measure Success
A national nonprofit that runs an after-school program in seven U.S. cities is shaking things up in the field through its unique youth-development model. The All Stars Project (ASP) is also working to develop a new metric for measuring the success of such programs.
The mission of ASP is to change the lives of young people through the power of performance.
Its after-school program, the All Stars Talent Show Network (ASTSN), features youth and young adults rapping, singing, and dancing.
ASP prides itself on not relying on the traditional model of after-school programs, which tends to pair academic work with some form of recreation.
Gabrielle Kurlander is the president and CEO of ASP.
"We don't believe that model any longer addresses the needs of the population," said Kurlander. "We developed a new model focused on bringing growth and development back into the equation. Young people need to learn how to learn, need to be exposed to a broader world in order to become successful adults."
The ASTSN provides an opportunity for students in the following cities to experience success on the stage:
- New York
- Newark, N.J.
- Bridgeport, Conn.
- San Francisco
- Jersey City, N.J.
Outreach teams for the program go into some of the most economically distressed communities in these cities and encourage young people to audition. Everyone who tries out makes the show. The program is open to students from kindergarten to 12th grade, and young adults up to age 25 are also welcome. In addition to performing, the participants also learn everything that has to happen behind the scenes for a production to work. For example, they staff the box office and serve as stagehands and technical staff.
The program, which is privately funded, also provides an opportunity for students to receive mentoring from local leaders through ASP's Development School for Youth. This outside-of-school program is for young people ages 16-21. It provides workshops on things such as resume writing and public speaking. It also places participants in paid summer internships. Some sites also feature special programs designed to develop leadership or improve relations between young people and the police. At the New York site, students also have the opportunity to learn about acting from theater professionals.
Most after-school programs are judged on their ability to help students improve grades, graduation rates or standardized test scores. But metrics like that can't measure the success of a program like ASP, which focuses on developing students in other ways.
In May of 2015, ASP began partnering with Southern Methodist University in Dallas to develop a new metric to judge after-school programs.
"We would like to see other after-school nonprofits have better tools to judge their work," said Kurlander. "We want funders to have a reliable tool or set of tools, so they can invest in the things that really make a difference."
Annie Wright is overseeing the project at SMU. She's the director of evaluation at the Center on Research and Evaluation (CORE) at the Annette Caldwell Simmons School of Education and Human Development.
"We discovered quickly with them that the kinds of outcomes that they wanted to measure the tools just really didn't exist for that yet, and so we have embarked on a process of developing customized evaluation tools that fit the All Stars model," said Wright.
She and her team have developed tools that measure four different domains of development:
- Relational capabilities (Can students create new relationships?)
- Vocational competence (professional dress, business etiquette)
- Positive self perception
- World perception (seeing themselves as part of a broader world).
The tools include a youth self-report survey for young people in the Development School for Youth that would be taken pre- and post-participation, a staff checklist for ASP workers to use to rate participants on the four domains of development, and an ensemble checklist that will gauge the overall quality of the group of young people a single youth is participating with, which would compare to ratings of classroom quality.
Her team is piloting these tools now, and in the next year or so will use them to begin collecting evaluation data.
Wright praises ASP for breaking new ground where after-school program evaluation is concerned. She says programs like ASP that focus on whole-child development and social and emotional learning are not well-served by metrics such as school attendance rates or reading scores.
"If we were to paint the All Stars Project and other organizations like it with that academic brush and just look at long-term academic outcomes, unfortunately, we'd miss a lot of really rich information about the mechanism of change and backfilling developmental and opportunity gaps that many youth face," Wright said. "By focusing on these proximal outcomes we're going to be doing a better job over time of determining what really works in out-of-school-time programming, and hopefully bottle that and keep doing it at a larger scale."
ASP hopes to have reportable results by 2019 with a final report slated to be released in 2020.
Photo: Six girls dance as part of ASP's All Stars Talent Show Network in Chicago. (Photo Courtesy All Stars Project)