« Name That Baby: Why 'Non-Cognitive' Factors Need a New Name | Main | Teachers Without Cages: Teachers and Their Talent for Problem-Solving »

Finding Needles in Rapidly Growing Haystacks: The Challenge of Data in Education

Note: Rick is taking a hiatus while he's off talking about his new book, The Cage-Busting Teacher. Meanwhile, this week's guest posts will be written by members of Panorama Education, a Boston-based startup that uses data analytics to help teachers and administrators improve their schools. Co-founder Xan Tanner (@xantanner) is guest posting today.

In 1869, Charles Joseph Minard drew what is widely regarded as one of the best graphs the world has ever seen. In one graph, Minard was able to highlight six of the most relevant pieces of context (location, distance, timeline, direction of travel, the number of troops, and the temperature) and the outcome of Napoleon's Russian Campaign of 1812 to tell a story that is far more powerful than the underlying data alone.

Minard.pngIt speaks volumes that even almost a century and half later, contemporary information designers the world over return to Minard's graph as the gold standard. The graph is a timeless example of the value of displaying only the most relevant information in the clearest form. Yet Minard had it easy: in that graph, he was likely displaying all the data he could get his hands on.

Today in education, we have more data than Minard could have possibly imagined. Just for an individual student, we collect data on whether and what time that student showed up to class, how the student did in that class at multiple checkpoints, and whether s/he was cited for behavioral issues, not to mention his or her student demographic characteristics and other background data. The list goes on and on, not just for students but also for teachers, administrators, and parents.

There is no question that this data is valuable: it represents great potential to help teachers, schools and districts throughout America. But there is an important truth in the field of analytics: just because the data was collected does not mean that it is particularly valuable or even usable in its current state.

Before helping to found Panorama Education with Aaron Feuer, I used to work on analytics for a college basketball team. Each week at the end of Wednesday practice, every player would receive the scouting report for the upcoming weekend's games. The scouting report was a guide chock-full of incredibly valuable data. Everything a player, coach, or fan could possibly want to know about the opposing team was in that document, down to each player's free throw percentage.

I was always tremendously excited about these reports. They represented the culmination of a week of hard work scouting, charting, and compiling information about the opposing team so that our team could do their job better. Yet time and again, despite my best efforts to put this information together and explain it, the players would not internalize the key material and would be unprepared for games.

They weren't prepared not because they weren't reading it or didn't value the information presented in the report—knowing another player's ability to hit free throws is a crucial piece of data for any basketball player in a late-game situation—but because there was too much data in the guides. What the coaching staff and I realized was that knowing that #24 had historically made 51% of his free throws did not help any player to play better. That level of specificity required a great deal of mental effort to process in stressful situations and was often useless in moments of quick decision-making. Instead, players needed the data in a different format. They needed to know that, when they needed to foul someone, they should foul #24.        

Similar situations happen all the time in education. We present tables of information to teachers, principals, and district administrators and leave them to parse complex contextual data in whatever tiny modicum of time they have. Unfortunately, this means that the majority of the time we as an education system spend with data is tracking and trying to understand the data we have collected in the past, as opposed to engaging with and taking concrete action steps to improve from that data.

As an education system, we need to shift our focus from data that merely represents past realities to data that actively helps us build better futures for schools everywhere. The standard for using data should not be to tell people at the end of a teacher evaluation process that they are a 3 on a scale of 1 to 4. Instead, the standard must be to provide the data that teachers need to go from being a good teacher to an outstanding one. The standard should not be providing all the data available to parents in a school report card: the standard should be to provide parents with the data they need to make the best possible choices for their child. Fundamentally, we should be measuring so many aspects of student outcomes and classroom experiences so that we know what to do next, not just to capture what's already happened.

For this reason, I believe the future of education data lies in figuring out how to get better at finding needles in rapidly growing haystacks. This responsibility does not only belong to people working with data in school districts; it is a responsibility shared by researchers, vendors, policy experts, educators, and other stakeholders that seek to use data to inform and improve the education system.

While this is by no means an easy challenge—every field and industry is struggling with some version of this problem—education has the potential to be far more successful at using data to improve outcomes because the results are not zero-sum. Stakeholders in all parts of education can work collaboratively, sharing key findings and best practices, to improve outcomes for students, educators and schools everywhere.

This potential for collaboration is the key thing to remember about the work with data we do in education. In basketball, there is only one victor at the end of a contest; data is used to give one side an advantage. In education, winning is defined by the impact we have on the lives of students and educators everywhere. In the end, that's what excites me most about the potential of analytics in education: if it's done well, we can all win.

--Xan Tanner

Notice: We recently upgraded our comments. (Learn more here.) If you are logged in as a subscriber or registered user and already have a Display Name on edweek.org, you can post comments. If you do not already have a Display Name, please create one here.
Ground Rules for Posting
We encourage lively debate, but please be respectful of others. Profanity and personal attacks are prohibited. By commenting, you are agreeing to abide by our user agreement.
All comments are public.

The opinions expressed in Rick Hess Straight Up are strictly those of the author(s) and do not reflect the opinions or endorsement of Editorial Projects in Education, or any of its publications.

Follow This Blog


Most Viewed on Education Week



Recent Comments