This month, the RAND Corporation released a new report, Informing Progress: Insights on Personalized Learning Implementation and Effects. This is a third report coming out of their study, funded by the Bill & Melinda Gates Foundation, of personalized learning implementation and student achievement in schools awarded grants through Next Generation Learning Challenges (NGLC, which also receives funding from the Gates Foundation). Our commentary on the first two reports is available in the posts Personalized Learning is Promising and Personalized Learning is (Still) Promising.
Before I dig into what RAND discovered, I’d like to share some perspective.
The personalized learning practices emphasized in the report are just one aspect of a broader set of goal-setting and learning practices in the NGLC community of next gen schools. Personalized learning is an important aspect of the schools’ models. But it doesn’t encompass the full vision the schools were designed to address: broader outcomes, more engaging and deeply powerful learning experiences, a restructured organization and use of resources, and plans for sustainability and scale. Check out our original Request for Proposals and Design Principles for more on the challenge NGLC issued to the field.
The findings are based on data collected in 2014-15. That’s two full school years ago. In an initiative that’s only five years young. The schools were in their first or second year of implementation in 2014-15. By design, they are engaged in continuous iteration and improvement as they learn more about what works and what doesn’t. And the lessons we have collectively learned in the past two years are numerous. The NGLC website offers 200 blog posts sharing many of those lessons.
The report gives us insight into NGLC schools’ early effects on student achievement in math and ELA, and it attempts to uncover what is unique or more prevalent when it comes to personalized learning practices. At the same time, it represents a snapshot into a discrete subset of practices in schools at the earliest stages of operation.
Key Findings
The graphic of key findings to the right is pulled from the report (p. 3). These reflections on their key findings are grouped by implementation and achievement effects.
Personalized Learning Implementation: RAND found it challenging to distinguish individual practices between the NGLC schools and the comparison group. Perhaps this is because personalized learning is just one aspect of next gen school design. Other factors are at play too. RAND observed that the NGLC schools were so new in 2014-15 that they were still rolling out all of the components of their model. It also appears that many schools in the comparison group were finding ways to give individualized support and tailored learning experiences to their students. That’s good news.
Importantly, the authors noted that a new, school-wide model can’t always be understood by looking at its parts: “Although advocates and reformers have developed PL models, many of the component practices are relatively common nationally, making it difficult to clearly identify what makes a school a PL school” (p. 40).
Effects on Achievement: RAND found that the degree of implementation of the personalized learning practices they were studying varied in NGLC schools. Trends in the achievement analyses--including increasing effects on achievement over two years of implementation--suggest to the RAND researchers that “greater implementation of PL practices may be related to more positive effects on achievement; however, this finding requires confirmation through further research” (p. 3).
Personalized Learning Practices
The report offers useful summaries comparing the 40 NGLC schools with a national comparison group. The summaries are organized around four elements of personalized learning: Learner Profiles, Personal Learning Paths, Competency-Based Progression, and Flexible Learning Environments. I’ll refer you to pages 9, 12, 16, and 20 to read them. The summaries identify the discrete practices that were stronger in the NGLC schools and which ones were equally used by the two groups. The summaries also describe challenges.
Here’s a sampling of the practices where the NGLC group was stronger:
- More NGLC teachers reported frequent receipt of high-quality student data and extensive use of the data to personalize instruction. (Learner Profiles)
- NGLC schools appeared to dedicate more time to one-on-one, tailored support of student learning. (Personal Learning Paths)
- Although most teachers in each group reported using competency-based practices, NGLC teachers and students reported higher levels of such practices. (Competency-Based Progression)
- Although both groups considered student grouping based on data to be a key strategy, NGLC teachers reported adjusting those groups more frequently. (Flexible Learning Environments)
The challenges of practice are also noteworthy for anyone looking to adopt personalized learning in their school or classroom. Here are just a few:
- Many NGLC schools struggled to use nonachievement data (e.g., behavior, attendance, socio-emotional skills) to inform instructional decisions and goal setting, in part due to challenges measuring those skills and integrating the results with other student data. (Learner Profiles; related quote to the right is taken from p. 11)
- Teachers perceived tension between offering student choice and the need to address standards. (Personal Learning Paths)
- Some teachers reported that organizing students into groups for the larger performance tasks could be difficult, because students were in different places in learning the material. (Competency-Based Progression)
Math and Reading Achievement
RAND found positive effects on one-year math and reading growth in the 32 NGLC schools that administered the MAP assessment: 0.09 in math and 0.07 in reading. The effects are small. Students in NGLC schools started the 2014-15 year with average math and reading achievement levels that were significantly below national norms. The growth in achievement didn’t catch up to the national norm.
In an analysis of achievement data from 16 NGLC schools that had two years of data, RAND reported cumulative growth that pushed students above the national norms in both subjects (the graphic below is from p. 35). The second year was important: “The largest gains on average appeared to occur in the second year, suggesting that PL systems may require some experience before operating at their fullest potential” (p. 35).
When looking at individual school effects, RAND found a great deal of variation, ranging from +0.35 to -0.44 in math and from 0.63 to -0.51 in reading. What inference to make of this variation is unclear, given that many of these schools were in their first year of implementation. But we should wonder about the differences between the highest and lowest performing schools.
RAND analyzed other subgroups too. Charter schools appeared to have larger effects on achievement than district schools, but RAND offered so many cautions about making any kind of conclusion about this particular result. Growth was seen across all achievement levels, from the lowest to the highest performers; “this suggests that PL is benefitting students at all ability levels.” And effects were largest for the middle school grades.
Recommendations for Personalized Learning Practice
In addition to recommendations for policy and philanthropy, RAND offered five recommendations for districts and schools (pp. 43-44). I found they echo many of the needs that we hear directly from educators and leaders in the NGLC community of schools:
- Provide teachers with the resources and time to pilot new instructional approaches and gather evidence of how well they work.
- Provide teachers with time and resources to collaborate on developing curriculum and on
reviewing and scoring student work. - Identify a school staff member (or two) who is comfortable with technology and has curriculum expertise to serve as a just-in-time resource for teachers.
- Provide resources and support for school staff to help them choose the most-appropriate digital or nondigital curriculum materials.
- Provide resources and support for school staff to integrate multiple data systems.
In their acknowledgements, the RAND researchers offer their gratitude “to the PL students, teachers, and administrators who voluntarily participated in project data collection” (p. 44). The NGLC team is grateful too--to those in NGLC schools and those who made up the national comparison sample. Each school community’s willingness to open its doors and allow all of us to learn together is what will bring better learning experiences to all students.
In an upcoming post, NGLC’s director, Andy Calkins, will provide his perspective on the findings of RAND’s study of personalized learning as it relates to the evolution of NGLC and the community’s understanding of next gen learning.