Rebooting MOOC Research
I have an article out in Science today, Rebooting MOOC Research, that summarizes some of my thoughts on how we could better take advantage of research opportunities from massive open online courses (MOOC) to advance the science of learning. The Science article is here behind a paywall, and the pre-print is on my site here (with a copy on its way to Harvard's DASH open repository). And if you like science but hate reading, the good folks at the Science podcast have me as their guest this week, and the HarvardX folks made a video summary.
In the article, I advocate for three shifts in MOOC research. (1) We need to study student learning rather than measuring student clicks, (2) we need to share data responsibly among instutitions so we can make cross-course comparisons rather than investigating single courses, and (3) we need to have more carefully designed experimental research to complement data mining and post-hoc observational research. [From these three points, you can figure out that my critique of MOOC research is that too much of it is post-hoc, observational analysis of participation data (clicks) from individual courses or small sets of courses.]
More than anything else, I argue that the existence of big data from MOOCs doesn't obviate the need for attention to research design: "Big data sets do not, by virtue of their size, inherently possess answers to interesting questions." Any breakthrough research that may result from large-scale online learning is much more likely to emerge from careful course design and experimental design rather than from sifting through the data left behind by courses after they have finished.
It won't be possible for researchers alone to make these changes. Complex experiments require multidisciplinary teams. We need to have a whole range of stakeholders pulling together to raise the bar for online learning research: course developers, universities, foundations, journal editors, scholarly conference organizers all have a role to play in ensuring that MOOC research advances the science of learning.
The article is short, just two printed pages, so there are lots of other ideas that didn't get into the article. I think we need much more qualitative research about how students are learning—and making meaning of their learning—off the platform and offline. Who do they talk with about courses? Who helps them when they get stuck? I think we should be doing much more to figure out how to implement features that would allow for more individual learning pathways through course materials. Despite their origins in computer science, most MOOCs aren't even making basic efforts to allow for different students to have different learning experiences (beyond encouraging students to figure out for themselves how to wander through material). We need more attention to how different students experience online learning, especially how students with different levels of preparation and privilege experience MOOCs differently. This runs directly into challenges with student privacy, which we also need much more research and thinking about. How do we balance the potential learning gains from data collection about individual learners with the risks of disclosing that data? Finally, I'm very interested in research being done on peer assessment, and how communities of learners can support each others growth in that way.
I remain cautiously optimistic that carefully watching how students act in learning environments (by digitally tracking their actions) can help us understand patterns of behavior that lead to richer, more lasting learning. I hope that low-stakes, large-scale learning spaces like MOOCs can be one place to advance that kind of learning research.
Lots of good questions to be diving into for a new year.