Systems Thinking: Lessons From 'Over the Counter, Under the Radar'
This post is by Susan Fairchild, chief knowledge officer for New Visions for Public Schools.
A couple of years ago the Annenberg Institute for School Reform released, "Over the Counter, Under the Radar." The report's authors, Toi Sin Arvidsson, Norm Fruchter, and Christina Mokhtar, examine the high school assignment patterns of the approximately 36,000 students in New York City who are either late enrolling or who miss the high school choice process each year. These students are labeled "Over the Counter" (OTC) and they represent high-need student populations in the city's school system. Arvidsson et al. found that OTC students are disproportionately placed in: 1) schools with higher percentages of lower performing students, 2) struggling high schools with much higher percentages of OTC students to begin with, and 3) high schools targeted for closure or that are being closed.
But it is this report's somewhat unexpected similarities to Sebastian Junger's "A Perfect Storm" that ought to give policymakers pause. The convergence of a disproportionate number of these high need, OTC students entering into a school where a pre-existing downward cycle of performance has already taken hold gives even more energy to this downward, self-reinforcing feedback loop. In other words, the effects of this type of district-level practice within schools that are already showing signs of vulnerability, further destabilizes them--in essence producing a perfect storm: a vicious cycle of downward performance that can be difficult to stop or reverse, and in some cases ending in school closure.
One of the unintended benefits of this report is the way in which it lends itself to a systems thinking interpretation. Three important systems thinking lessons emerge from the "Over the Counter, Under the Radar" report that are crucial to consider if we hope to avoid these types of policy missteps in the future and if our ultimate goal is to fully support our educators and our students.
Lesson No. 1: We need to be more curious about a system's structure.
A system's structure generates its behavior. Barry Richmond defines systems thinking as "the art and science of making reliable inferences about behavior by developing an increasingly deep understanding of underlying structure." Systems thinkers are trained to look at the structures (e.g., how departments are organized, how students are programmed for courses), the interdependence of those structures, and the ensuing dynamics of those structures. In the case of "Over the Counter, Under the Radar," it is the "success to the successful" archetype that is fueling the inequities at the extremes that Arvidsson, et al. highlight in their report. Chris Soderquist and I have written at length about this here and the systems thinking structure can be found in this deconstructed map located here. Also, consider reading Chapter 3 of Tony Bryk's new book, Learning to Improve. Tony and his co-authors devote an entire chapter to "seeing the system."
Lesson No. 2: We need to know something about feedback loops.
Feedback loops are at the core of a closed-loop system (e.g., x causes y, which then causes x) and thus a critical concept in systems thinking. Feedback represents a process where "an initial cause ripples through a chain of causation ultimately to re-affect itself." In other words, feedback is both cause and effect.
There are two types of feedback loops: balancing and reinforcing. Balancing loops stabilize a system, as Donella Meadows writes, and oppose "whatever direction of change is imposed on the system. If you push a stock [an accumulation of something you can see, feel, count or measure] too far up, a balancing loop will try to pull it back down. If you shove it too far down, a balancing loop will try to bring it back up." The data patterns that are generated when a balancing loop is running the system are often oscillatory. Reinforcing loops, on the other hand, are "runaway loops." Meadows notes that reinforcing loops have an amplifying or snowballing effect, causing virtuous cycles that generate healthy growth or vicious cycles that are responsible for runaway destruction. The data patterns that signal the presence of a reinforcing loop are exponential growth or decay.
In "Over the Counter, Under the Radar," it is a reinforcing loop that is the dominant mechanism that further destabilizes low performing schools. What this means is that low performing students in the school beget future low performing students who will enter the school. That is, the population of lower performing students in a school has the ability to grow "as a constant fraction of itself." In system's thinking terminology, as the stock (the accumulation) of low performing students increases, the flow (the input mechanism that causes the stock to grow) gets bigger, which then makes the stock bigger. In Figure 1, the reinforcing loop is represented by the red line connecting the stock back to the flow and denoted by the R. This stock is "re-affecting" itself. And without any intervention, this population will likely grow by some rate each year (the growth rate).
This reinforcing feedback loop is already at work in many lower performing schools. What turns this vicious cycle into a perfect storm are the mechanisms that increase the growth rate. When OTC students are disproportionately assigned to low performing schools, the growth rate increases (see Figure 2). This practice accelerates the exponential growth of low performance. In other words, it's like turning the faucet of the flow "entering low performing students" hard to the right.
Lesson No. 3:"Doing and undoing have fundamentally different time constraints"
Practices and policies change the system in ways that are often unforeseen. For instance, the amount of time it takes to fill up a stock of low performing students is not equivalent to the amount of time it takes to drain a stock of low performing students. Some stocks can take very little time to fill, but can take incredibly long times to drain. In the stock and flow map in Figure 3, Flow 1 is an easy flow to increase as we see in the practice of assigning a disproportionate amount of OTC students to lower performing schools. Flow 1 is the result of actions taken by individuals outside of the school. Flow 2, on the other hand, is much, much harder to increase and is the responsibility of a different set of individuals--the educators within the school. Substantially more resources are required to move lower performing students into higher performance categories (Flow 2).
And at the same time, educators have to be worried about Flow 3. Systems are anything but static and just because a student is high performing at one moment in time does not mean she will continue to maintain her high performance. The stock of high performing students is not particularly easy to fill--but is easy to drain relative to the stock of lower performing students (See Figure 4).
Therefore, draining the stock of lower-performing students means that educators must open up Flow 2, the movement of lower-performing students into higher performance categories while simultaneously shutting off Flow 3, the flow of high performing students draining into the lower performance category. Given the effort that has to go into Flows 2 and 3, it becomes easier to see how district-level practices that exacerbate Flow 1 undermine our educators who are attempting to stabilize a fragile school environment.
These three system properties have both policy and practice implications. When we introduce policies into a school's ecosystem without giving consideration to feedback loops (both reinforcing and balancing) already at work within a school, we may unknowingly do harm (as in the case of the disproportionate assignment of high-need students to high-need schools documented in the Annenberg report). We should also consider how one policy intersects with and potentially counteracts other policies running concurrently within a school. At the time of this report, there is no question that different policies and interventions were running in low performing schools for the purpose of improving them. But, the disproportionate assignment of high-need students to high-need schools, as reported in the "Over the Counter, Under the Radar" report, possibly negated many other policy and intervention efforts that were designed to improve performance within those same schools. This suggests that when policies emerge from one area within a large system like New York City's education system without awareness of another group's policies that are simultaneously being introduced (or are already in place), we unknowingly undermine policy effectiveness. Undoubtedly, there are any number of costs associated with introducing countervailing policies within a school's ecosystem.
Dynamic system attributes also have implications for school practice. Schools can no longer afford to be vulnerable to the different needs of students who walk through their doors. Workflows that allow schools to quickly adapt to changing student need are imperative, otherwise opportunities to help all students reach their potential are quickly missed. Educators must get serious about systems if they hope to optimize and shape student and school trajectories.