Finding Needles in Rapidly Growing Haystacks: The Challenge of Data in Education
Note: Rick is taking a hiatus while he's off talking about his new book, The Cage-Busting Teacher. Meanwhile, this week's guest posts will be written by members of Panorama Education, a Boston-based startup that uses data analytics to help teachers and administrators improve their schools. Co-founder Xan Tanner (@xantanner) is guest posting today.
In 1869, Charles Joseph Minard drew what is widely regarded as one of the best graphs the world has ever seen. In one graph, Minard was able to highlight six of the most relevant pieces of context (location, distance, timeline, direction of travel, the number of troops, and the temperature) and the outcome of Napoleon's Russian Campaign of 1812 to tell a story that is far more powerful than the underlying data alone.
It speaks volumes that even almost a century and half later, contemporary information designers the world over return to Minard's graph as the gold standard. The graph is a timeless example of the value of displaying only the most relevant information in the clearest form. Yet Minard had it easy: in that graph, he was likely displaying all the data he could get his hands on.
Today in education, we have more data than Minard could have possibly imagined. Just for an individual student, we collect data on whether and what time that student showed up to class, how the student did in that class at multiple checkpoints, and whether s/he was cited for behavioral issues, not to mention his or her student demographic characteristics and other background data. The list goes on and on, not just for students but also for teachers, administrators, and parents.
There is no question that this data is valuable: it represents great potential to help teachers, schools and districts throughout America. But there is an important truth in the field of analytics: just because the data was collected does not mean that it is particularly valuable or even usable in its current state.
Before helping to found Panorama Education with Aaron Feuer, I used to work on analytics for a college basketball team. Each week at the end of Wednesday practice, every player would receive the scouting report for the upcoming weekend's games. The scouting report was a guide chock-full of incredibly valuable data. Everything a player, coach, or fan could possibly want to know about the opposing team was in that document, down to each player's free throw percentage.
I was always tremendously excited about these reports. They represented the culmination of a week of hard work scouting, charting, and compiling information about the opposing team so that our team could do their job better. Yet time and again, despite my best efforts to put this information together and explain it, the players would not internalize the key material and would be unprepared for games.
They weren't prepared not because they weren't reading it or didn't value the information presented in the report—knowing another player's ability to hit free throws is a crucial piece of data for any basketball player in a late-game situation—but because there was too much data in the guides. What the coaching staff and I realized was that knowing that #24 had historically made 51% of his free throws did not help any player to play better. That level of specificity required a great deal of mental effort to process in stressful situations and was often useless in moments of quick decision-making. Instead, players needed the data in a different format. They needed to know that, when they needed to foul someone, they should foul #24.
Similar situations happen all the time in education. We present tables of information to teachers, principals, and district administrators and leave them to parse complex contextual data in whatever tiny modicum of time they have. Unfortunately, this means that the majority of the time we as an education system spend with data is tracking and trying to understand the data we have collected in the past, as opposed to engaging with and taking concrete action steps to improve from that data.
As an education system, we need to shift our focus from data that merely represents past realities to data that actively helps us build better futures for schools everywhere. The standard for using data should not be to tell people at the end of a teacher evaluation process that they are a 3 on a scale of 1 to 4. Instead, the standard must be to provide the data that teachers need to go from being a good teacher to an outstanding one. The standard should not be providing all the data available to parents in a school report card: the standard should be to provide parents with the data they need to make the best possible choices for their child. Fundamentally, we should be measuring so many aspects of student outcomes and classroom experiences so that we know what to do next, not just to capture what's already happened.
For this reason, I believe the future of education data lies in figuring out how to get better at finding needles in rapidly growing haystacks. This responsibility does not only belong to people working with data in school districts; it is a responsibility shared by researchers, vendors, policy experts, educators, and other stakeholders that seek to use data to inform and improve the education system.
While this is by no means an easy challenge—every field and industry is struggling with some version of this problem—education has the potential to be far more successful at using data to improve outcomes because the results are not zero-sum. Stakeholders in all parts of education can work collaboratively, sharing key findings and best practices, to improve outcomes for students, educators and schools everywhere.
This potential for collaboration is the key thing to remember about the work with data we do in education. In basketball, there is only one victor at the end of a contest; data is used to give one side an advantage. In education, winning is defined by the impact we have on the lives of students and educators everywhere. In the end, that's what excites me most about the potential of analytics in education: if it's done well, we can all win.