This week we are hearing from the Research Alliance for New York City Schools (@RANYCS). This post is by Kayla Stewart, Communications Coordinator, Lisa Merrill, Research Associate, and Chelsea Farley, Communications Director at the Research Alliance.
Today’s post is written from the researcher perspective. Stay tuned: Thursday we will share the practitioner’s perspective on this research.
Why This Collaboration
In recent years, educators, researchers, and policymakers have placed a greater focus on measuring and developing students’ abilities beyond traditional academic skills. Evidence suggests that social-emotional learning (SEL) is also critical for student success. This includes a number of mindsets, attitudes, and skills that enable young people to have positive relationships, manage emotions, set and pursue goals, and overcome obstacles.
In 2015, the Student Success Network approached the Research Alliance for NYC Schools for assistance with their strategy for measuring SEL outcomes. The Student Success Network (SSN) is a group of about 50 youth development and education organizations in New York City that are working together to measure and improve students’ SEL skills. SSN organizations serve more than 150,000 middle and high school students throughout the City, providing a diverse array of programs. They have chosen to measure eight specific SEL learning factors through a student survey: Growth mindset; academic self-efficacy; interpersonal skills; self-regulated learning; enlisting social resources; self-advocacy; problem solving; and belonging.
Figure courtesy of the Student Success Network
The collaborative work between the Research Alliance and SSN has since developed into a robust partnership, focused not only on building more valid and reliable SEL measures, but also on helping member organizations utilize the information that the survey elicits as part of a continuous improvement process.
What The Collaboration Worked On
Prior to our collaboration, SSN had identified SEL competencies they wanted to measure and created their own student survey. The survey was being administered to students at individual sites across the network twice a year (once in the fall, and once in the spring). When our work together began, the research team realized SSN was not tracking response rates for the surveys, making it difficult to judge the quality of the resulting data. Research Alliance and SSN staff worked together to develop a system to track response rates, and found they were initially quite low. SSN leaders and member organizations then worked to implement strategies that would encourage more students to complete the survey. Since then, response rates have steadily improved--an essential change that is helping ensure the survey is more representative of network sites, programs, and students.
“Lisa [Merrill, Research Associate at the Research Alliance] has pushed practitioners to think about the quality of the data they are collecting and was instrumental in our push to track response rates,” said Alex Lotero, Director of Data Strategy at SSN. "[For example], after listening to the difficulties of gathering active consent from parents, she submitted an amendment to our NYC DOE IRB agreement to allow our organizations to use passive consent.”
The Research Alliance and SSN also worked together to refine specific survey measures and to develop methods for using the survey data to identify promising program strategies. This involved creating clusters of comparable program sites (with high response rates). We then used the fall and spring survey results to identify “Bright Spot” program sites whose students had experienced more SEL growth than other students in their cluster. This approach allowed us to make apples-to-apples comparisons and identify sites that had outperformed other, similar programs. (Simply looking at sites with the highest scores in the spring or with the largest growth would have been problematic, given the diversity of programs and student populations across the SSN network).
SSN used the information from this analysis to engage in conversations with their members about which practices might be responsible for the exceptional growth in SEL skills seen at the “Bright Spot” sites. SSN and the Research Alliance are planning to test some of these theories and identify new potential “Bright Spots” in future work. Indeed, there is a great deal more to be learned from the data we have compiled, which includes extensive information about students’ academic progress (e.g., grades, attendance, test scores, credit accumulation), as well as their SEL outcomes.
Implications For Practice
While our collaboration is ongoing, our work to date has already produced a stronger survey that is yielding higher-quality data. More importantly, it has fostered organizational change within SSN organizations. This research-informed approach to continuous improvement is supporting organizations throughout the City as they work to identify and implement effective practices for improving students’ SEL skills.
The practitioners and researchers involved in this work have been open, accessible, humble, and curious—ingredients that we agree are essential for a thriving research-practice partnership.
“Researchers bring an eye for measurement design, data quality, and rigorous analysis and vetting of findings or results,” said Lotero. “Practitioners bring a knowledge of on-the-ground realities, problems of practice, and deep although oftentimes anecdotal knowledge of student needs. A strong partnership results in the use of data to improve practices and student experiences, documentation of learnings, the building of an evidence base for practices, and high-quality data collection using research-based tools paired with rigorous analyses.”
SSN’s commitment to integrating research throughout the continuous improvement process has enabled the Research Alliance to learn more about measuring and improving SEL skills—and to partner with practitioners who can directly and continuously apply research results within their organizations. We think this partnership holds great promise for better serving the many tens of thousands of students who are involved in SSN member programs.
Please visit our website for more information about our collaboration with the Student Success Network.
Previous blog posts by the Research Alliance for New York City Schools:
Inequities in College Access and Success: Findings From New York City
Beyond Tuition: Needs-Based Grants Can Close Racial Attainment Gaps
4 Key Strategies for Improving School Culture for Black and Latino Male Students
Curious about other research topics partnerships have written about for this blog? See this Guide to the NNERPP EdWeek Blog for all previous blog posts organized by research topic area to easily find other posts of particular interest to you!