In Education, Bringing the Dark Matter Into the Light
Does the rise of big data augur anything positive for American public education? After reading a new article by celebrity statistician Nate Silver, I think the answer is yes - but only if we're willing to reconsider the meaning of that single word - data - and not just by acknowledging how a more strategic use of information can help children learn, but also by admitting that much of the mystery around how children learn and grow will remain just that - a mystery. Of course, the Silver story does not appear in Education Week, but ESPN Magazine, and it's not about personal growth, but professional sports. What he's chronicling is the increasing use and effectiveness of analytics in sports like baseball and basketball, and the ongoing reluctance of professional football to follow suit. "From studies on pitch framing to attempts to measure clubhouse chemistry," Silver explains, "topics once thought to be beyond the realm of analytics - the Dark Matter - now seem within reach." Any sports fan knows what Silver is talking about. When I was a kid, the stats that adorned the back of my baseball cards - batting average, home runs, etc. - were the ones we tracked in little league to gauge our own worth. Today, however, sports stats like AVG and PPG have given way to analytics like WAR and PER - wins above replacement and player efficiency rating, respectively - resulting in revolutionary adjustments to how the games are played. In professional basketball, for example, it's now common knowledge that one shot in particular - a three-pointer from the corner - produces an average of almost 119 points per 100 possessions. "How impressive is that? It's roughly equivalent to a dunk or layup attempt from the restricted area - or to Michael Jordan's career statistics." In a multibillion-dollar industry like the NBA, you can see why that sort of insight is worth almost any investment to uncover. And when you consider the ways in which human behavior is increasingly making itself available to serious number-crunching (Gmail, anyone?), you can see why the search for the statistical equivalent of the "Corner 3" would bleed into lots of other industries, including, yes, public education. When I think about data-driven decision-making in schools, however, I'm drawn less to the K-12 equivalent of WAR (though it would be cool if there was one), and more to Silver's reference to Dark Matter - literally, a theoretical type of matter that makes up the bulk of the universe, and figuratively, a phrase to represent the still-considerable blind spots that remain, even in the age of big data. And I'm discouraged by the considerable chasm that has developed between the opposing armies of the education reform debate - one of which would like us to simply leave all that Dark Matter alone and unobserved, and the other of which would like us to commit ourselves full-time to leaving no stone unturned. Indeed, whereas new-school organizations like the Gates Foundation want to engender an analytical approach to teaching and learning that rivals the NBA's approach to building a championship team, old-school entities like the AFT are urging us to adopt the human-centered approach of the NFL, which has chosen custom and consistency over what the data (and diehard fans) have been telling them to do for years - like go for it on Fourth Down. Last season, for example, "NFL teams kicked or punted when they should have gone for it 693 times, or 21.7 mishandled situations per team." How much difference does this make, Silver asks? About half a win over the course of a sixteen-game season. "That might not sound like much," he writes. "But 0.5 wins over a 16-game football season is equivalent to five wins over a 162-game baseball season." And how much does five extra wins - or WARs - get you if you play baseball for a living? About $130 million. Personally, I'm in between the Gates folks and the AFT folks on these issues. If there are new ways in which we can diagnose more precisely what children know and don't know, we should apply that wisdom ASAP in the service of helping children learn and grow. And we must remember that for the foreseeable future, it's the Dark Matter that will dominate - which means we should stop pretending that what we currently measure is in any way a legitimate reflection of what we want to be measuring. Unfortunately, there aren't many people inhabiting the middle ground, but there should be. Look at what's happening at Gallup, for example, where they've created a survey to help adults better gauge the hope, engagement and well-being young people bring with them to school each day. (Maryland's largest school district is even using it as part of their official decision-making process). Or check out a place like the MC2 School in New Hampshire, which has identified the core skills and dispositions its young people will need to navigate life - and outlined the measurable steps it will take to get them there. What these efforts demonstrate is that we've already begun, inexorably, to swing the pendulum away from the myopic overreliance on reading and math scores, and towards something more holistic, more reflective of what we truly value, and more useful to both teachers and students in their day-to-day work. Because the pendulum swings slowly, however, a lot of educators, for good reason, have become war-weary of any new effort to measure their work. The result is a growing army of resistance that has taken to Twitter to provide examples of the immeasurable aspects of teaching and learning - the Dark Matter, if you will - along with a rhetorical hashtag challenge -- #evaluatethat. The implication is that you can't measure such things - ever. But it seems to me the goal should be to create systems in education that do account for the deeply human, seemingly unquantifiable things - and not in an effort to quantify and incentivize them, but to create better alignment between what we know is in the best interests of children, and how we go about evaluating our own ability to meet those needs. As Nate Silver points out, "all the data in the world is for naught if it's not accompanied by a culture that empowers players, coaches and general managers to make better decisions." The same is true for our schools. Better information about what is actually happening to children, and what is causing them to thrive, should be a goal we can all embrace - but only if we're willing to acknowledge both the ubiquity of the Dark Matter in the field, and the need to bring more of the mystery into the light. Follow Sam on Twitter.