As families, educators, and community leaders wrestle with COVID-19, we’ll be trying to bring conversations to readers that will be helpful in confronting the challenge.
My good friend and colleague Nat Malkus, data aficionado and deputy director of Education Policy Studies at AEI, has just launched a nationally representative survey that tracks school district responses to the coronavirus pandemic as it unfolds. I had a chance to chat with Nat about that survey and how he sees it being useful during this time of crisis and uncertainty. Here’s what he had to say.
Rick: So what exactly is this thing you’ve built anyway?
Nat: When the pandemic upended school for 50 million students in a matter of weeks, I wanted to know how districts across the nation were going to retool. But more than that, I knew that this unprecedented problem would evolve each week as educators would essentially have to build this plane as it is going down the runway, so I wanted to develop a survey that could capture that evolution. So we built a small, nationally representative survey which we call the COVID-19 Education Response Longitudinal Survey (C-ERLS) to match four criteria. We wanted to make it rapid—able to capture data in a two-day window; reliable—collecting information from district websites which are the hub of their communication about schools’ transitions; representative—reflecting change in districts across the country; and repetitive—conducted every 7-10 days as remote instruction develops.
Rick: What sparked the idea?
Nat: Honestly, it flows from advice I heard the note economist Alan Krueger had given, “If you don’t have the data you need, go get it!” I had been looking for nationally representative data on what schools and districts were doing and I could not find what I was looking for. So on March 23rd, I put pen to paper, sketched out a survey instrument and a plan to employ it, pulled a team together, and drew a sample of 250 districts. On March 26th, we started data collection. The real motivator for this speed was the pace of change in districts and the knowledge that if we didn’t get a survey set up quickly, we would miss the front end of the response.
Rick: So I actually just chatted with Robin Lake about CRPE’s data collection last week, and Ed Week is obviously tracking a lot of this stuff. How is what you’re doing here different?
Nat: C-ERLS is definitely a complement of stellar work being done elsewhere and fills some important gaps. CRPE’s district tracker, which you highlighted last week, is similar to ours in a number of ways. They have great information on large districts and also cover a number of larger charter-management organizations, and they are also expanding to a nationally representative sample of 400 districts, more than we have now. Education Week has good coverage at the state level, and is tracking things over time.
We complement and add to these by having the first nationally representative sample of public school districts, which provides a look at what districts across the nation are doing—big, small, rural, urban, etc.—adding important information to the changes in big cities and well-known districts that have received more coverage. That complete picture is information that folks are interested in right now.
We also designed C-ERLS to clearly track how those same districts change over time. This is important because it will provide a national look at how school and district efforts evolve as the pandemic plays out. Taken together, these complementary projects will provide a much better sense, at multiple levels, of how the public education system is retooling under pressure.
Rick: OK, so who is this most useful for? And how do you envision folks using this tool?
Nat: First off, C-ERLS will be useful for journalists and the public who are trying to get a sense of the landscape of public school offerings amidst this tumult. I also think it will be useful for educators and district leaders who may want to know how what they’re doing fits into the efforts we see across the board. I have even gotten some interest in what we are finding from federal officials who are looking to understand the challenges in schools and districts, and be responsive, but are desperate for good vantages to look from.
Rick: Can you talk a bit about the data collection? Who is doing it? How often? How did you design the survey instrument? How did you get your sample? Etc.
Nat: Sure, and the key at each of those aspects has been managing tradeoffs. Our education policy team at AEI is collecting data in house, having added—meaning begged, borrowed, and stolen—staff from other AEI teams to get a total of 15 trained data collectors. Our first survey wave closed on March 27, and the second wave started 10 days later. We will keep doing surveys every 7-10 days, as events warrant. I quarterbacked the survey design, trying to keep it to a manageable size for repeated collections, while still tracking an array of important aspects from closure timeframes to the provision of meals, technology, and student accommodations. Of course, the focus is to capture the kinds of remote educational approaches districts and their schools are adopting.
Rick: What have you learned so far? Are there any standout, creative ideas that have you excited?
Nat: The most amazing thing I have learned, which I knew but had not quite digested, is how fast all this happened. We found that 65 percent of schools in the districts we surveyed closed between March 16 and 18. That’s three days! The pace of change in schools cannot be overstated.
Providing meals was a first-order challenge for many districts, and 82 percent had meals up and running on March 27. Small districts (whose identities we are masking) were stepping up, with one providing meal pickup at all its schools and several bus stops, including three breakfasts and lunches on Fridays. Several other districts developed the capacity to deliver meals to families’ front doors, when needed.
Some districts’ efforts have seen a bit of two steps forward, one step back, in response to conditions on the ground. For instance, the Dearborn public school district (Mich.) was loaning out one Chromebook per family for remote instruction. However, due to Michigan’s “stay at home” order, they have suspended distributing Chromebooks, leaving some students stranded.
As for the different ways districts have responded on instruction, they cover the gamut, from offering nothing until they have had time to formulate a developed plan, to rapid deployment of new systems connecting students with all their teachers. Many are using Google Classroom or other asynchronous online platforms, many of which predated the pandemic closure and are now getting far more traffic.
Others are starting fresh. One small district we saw had developed a system on Google Docs where students and parents could click through a listing of schools, to every teacher, to every class, which allowed teachers to easily update content and students to pull down work. That may not be the most efficient route, but it was up and running in less than two weeks and shows how determined teachers, principals, and district staff will find ways to reach their students.
Finally, districts are communicating with their communities in appropriate ways that are both sobering and heartening. For instance, the Detroit Public Schools Community District maintains a list of those in their community who have been infected with COVID-19, as well as those who died in this tragedy. One of our small districts is keeping its communities spirits up with principals posting daily videos that include announcements, birthday shoutouts, and clips of students leading the Pledge of Allegiance.
Rick: Looking ahead, what do you hope will come out of this project?
Nat: Beyond the current view that these data provide, we structured the collection so scholars might make use of it in more depth in the future. Next year, or the year after that, untold analyses and dissertations will cover schools’ pandemic response, so we are archiving every web page, pdf, document, and video that we examine in each wave for each district. As these web pages are updated from week to week, we don’t want the information documenting it on the web to be lost, so we are saving and indexing these files in hopes they are used in the future. (And I hope they are used, because archiving them is a pain, and its only wave 2!)
Rick: OK, last question. If you could give one piece of advice to districts based on what you’ve seen, what would it be?
Nat: I am reticent to give advice when schools are in such uncharted waters. But since you asked, I’ll suggest advice on two sides of the same coin. First, districts are working rapidly to adjust, but they are bound to be in those uncharted waters for a long sail, to extend the metaphor. Careful development of platforms that are both productive and sustainable, for teachers and students, are probably worth a little extra time on the front end. At the same time, kids and families are in for a long sail, too, and the human connections students depend on may be the hardest aspect of schools to re-create in remote learning. Ensuring schools deliver on those needs wasn’t addressed concretely on many district websites, and I want districts to make sure they maintain educational and personal connections to students while they are away.
This interview has been condensed and edited for clarity.