As Diploma Plus expanded its network of schools to support students at risk of dropping out of school, it developed a series of data tools to ensure that each of its locations was tracking progress and providing the services each participant needed. It now serves over 3,400 students, and uses the tools it has developed to inform program improvement and professional development. The program, based in Boston, is one of three working with disconnected youth around the country that is being highlighted in a new report about the role data can play in improving and expanding services.
The report, entitled Beyond the Numbers: Data Use for Continuous Improvement of Programs Serving Disconnected Youth, done by the American Youth Policy Forum, explores the use of data in programs that serve out-of-work and out-of-school youth between the ages of 14 and 24, who are often called “disconnected youth.”
The analysis consisted of case studies of three such programs: Roca, an organization in Massachusetts that seeks to move high-risk youth into educational, employment, and life skills programming; Our Piece of the Pie, a youth development agency in Hartford, Connecticut that helps young people access educational, employment, and personal skills; and Diploma Plus, a national network of public schools for at-risk students.
The study points to three main indicators of success shared among the organizations. The first indicator is a “clearly articulated theory of change,” or an outline that shows what the organization would like to change, how it will make the change, and what the outcome will be.
For example, the organization Our Piece of the Pie uses the web-based software Efforts to Outcomes to track participant data based on a set of outlined short-term, intermediate-term, and ultimate outcomes. These outcomes are reached through the building of relationships between participants and youth development specialists. “We know that what makes a difference in people’s lives and what makes them change and move forward is the relationships they build,” said OPP President and CEO Bob Rath in a webinar on Tuesday. The relationship-based model is also a key element of Roca’s theory of change.
The second common element of success identified in the study is that the data used reflects the mission of the program. Roca, for instance, which also uses the Efforts to Outcomes software, excludes any data that is not relevant to its three-phase “High Risk Intervention Model,” and “capable of providing funders a complete view of its impact on youth, while at the same time, preserving ETO as a valuable and functional database for staff,” according to the report.
The third shared indicator of successful use of data is that the organization shows a commitment to professional development among staff members who work with and collect data. Diploma Plus designates “implementation coaches,” who are responsible for collecting data and engaging members of their school communities in professional development on creating performance-based assessments.
Bob Rath agrees with this method of transparency in data among staff members. Programs must “use data as a flashlight, not as a sledgehammer,” he said. He often finds it can be helpful to have a “written agreement in terms of data sharing.”
Roca not only shares data with staff members, but also with police officers. “One of the things that has really helped me,” said director of programming Dana Betts, “is getting the police to know actually what we’re doing and the work that we’re doing.” This gives police the opportunity to see when young people who may have had a history with crime are making progress.