For the current issue of Education Week, I wrote about University of Wisconsin-Madison researchers who believe video games could revolutionize the assessment field. I had way too much thought-provoking material to cram into 1,500 words...here’s Part 2 of what got left out. To read more about video games as “petri dishes” for assessment, check out Part 1.
__________________________________________________________________________________
Grit. Tenacity. Self-control.
Since publication of Paul Tough’s influential book How Children Succeed, everyone seems to be talking about these so-called “noncognitive skills.” The research is compelling: children’s ability to focus their attention, delay gratification, and persevere past obstacles is highly correlated with everything from wealth to physical health later in life. There are also indications that the parts of the brain that regulate these noncognitive skills remain highly malleable well into adolescence, offering hope that timely, thoughtful interventions can make a big difference in kids’ lives.
But how can educators measure something like a child’s resilience? And should they try?
It’s a challenge that U.S. Secretary of Education Arne Duncan calls “the next frontier in assessment,” and the federal government is tentatively beginning to nudge researchers and industry leaders to develop the tools that can make it happen.
The U.S. Department of Education’s Office of Educational Technology, for example, is currently taking public comments on a draft of a forthcoming report, Promoting Grit, Tenacity, and Perseverance: Critical Factors for Success in the 21st Century.
“The test score accountability movement and conventional educational approaches tend to focus on intellectual aspects of success, such as content knowledge. However, this is not sufficient,” contends the draft report. “To design and evaluate learning environments that effectively promote and/or teach grit, tenacity, and perseverance, the field will need valid and reliable measurement instruments that can provide quick and useful feedback.”
In other words, the federal government believes we need new ways to gauge students’ noncognitive skills. And they expect technology to play a big role, soon.
“It’s certainly an emerging field,” Richard Culatta, the director of the Office of Educational Technology, told Education Week. “It’s a call that [Secretary Duncan] is clearly making and that we’re trying to support.”
More High-Tech Solutions
Right now, the most prominent existing example of an attempt to assess students’ noncognitive skills is probably KIPP’s “character growth report” (recently re-branded from its earlier incarnation as a “character report card.”) Relying mostly on what are known in the assessment field as “informant reports” provided by teachers, KIPP is experimenting with giving the students in a handful of its New York City charter middle school scores for character traits such as “zest,” “grit,” and “social intelligence.”
“KIPP schools around the country are now focused on how we can integrate a more structured and measurable approach to character development,” according to the group’s website.
But as the Office of Educational Technology’s draft report points out, relying on such low-tech methods as teacher ratings, student self-reports, or even school records comes with a host of problems, ranging from possible bias to lack of precision.
The report highlights a couple of new technologies that could help fill in the gaps and provide a more nuanced, well-rounded picture:
- Educational data mining and learning analytics, in which massive amounts of digital data on students are collected and statistically analyzed to look for evidence of learning or growth. For example, let’s say a child is confronted with a challenging math problem online. How long does that child wrestle with the problem? How many different ways of trying to solve the problem does the child attempt? How distracted does the child get? All of that information can now be automatically tracked and correlated with other learning and life outcomes.
- Affective computing, in which “systems and devices that can recognize, interpret, process, and simulate aspects of human affect” are used to gauge students’ emotional and physiological responses. So, in the scenario above, does a child exhibit physical signs of becoming engaged or frustrated when confronted with a challenge? Do they smile, sweat, get increased blood pressure? All of that information could soon be gathered and used to gauge whether students are developing the character traits that research now says are the building blocks of success.
Though not mentioned in the report, video games are another potentially powerful tool to assess noncognitive skills.
Researchers at the University of Wisconsin-Madison are currently testing Crystals of Kaydor and Tenacity, two iPad games designed to simultaneously cultivate and assess students’ empathy and ability to pay attention. As I wrote for our latest issue, the project is way out on the cutting edge of this nascent new approach to assessment.
In Crystals, players assume the identity of a damaged robot stranded on a distant planet. To repair themselves and return home, they must successfully interact with human-like aliens, recognizing and responding to the emotions the creatures display on their faces. Correctly identifying and “calibrating” the aliens’ emotions is intended to be a measure of players’ empathy, operationalized in the game as the ability to pick up on other’s nonverbal cues.
The idea of developing and assessing empathy through interactions with digital aliens might sound a bit out there, but the researchers behind Crystals believe that it’s both possible and a good idea.
To start, the aliens in the game aren’t just cool-looking. Artists in Wisconsin’s Games+Learning+Society center, along with scientists in the university’s Center for Investigating Healthy Minds, spent months painstakingly animating the creatures’ human-like faces, working and re-working the digital aliens until they could depict human emotions like fear and surprise in ways that could be scientifically validated via what’s known as the Facial Action Coding System.
And neuroscientist Richard Davidson, who directs the Center for Investigating Healthy Minds, argued that empathy is one of the most important skills that young people can develop.
“There’s a vast array of scientific literature that suggests that in both the animal kingdom and in humans, the ability to read the facial expressions of others is actually a critical adaptive skill,” Davidson said. “It’s not a value judgment. It’s necessary for survival.”
In recent years, there’s been a tidal wave of new research providing empirical evidence that noncognitive skills are at least as important as IQ in determining individuals’ life outcomes.
Just as important, Davidson argued, is that the circuits in the brain that regulate traits like empathy, paying attention, persevering, and focusing the mind are “highly plastic” and thus comparatively easy for people to improve well into adolescence.
The result, he said, is a window in which educators and parents—and video games—might be able to make a big difference. “If we can intervene with children and provide them with experiences and training procedures that actually strengthen the circuits in the brain that are beneficial for life outcomes,” Davidson said, “I think we have a moral obligation to try.”
Video Games as Standalone Assessments?
Davidson said his primary interest is in the extent to which games like Crystals can help strengthen children’s noncognitive skills.
His colleagues in Games+Learning+Society are equally interested in the extent to which such games might eventually function as a new form of standalone assessment. The idea is that playing games like Crystals or Tenacity would simultaneously develop and measure students’ empathy or ability to pay attention, all in real time, all in the context of an activity that they actually enjoy doing.
“It’s this ongoing, formative type of assessment where we’re actually able to see where learners are, and how they’re performing, and what they understand and don’t understand and can and cannot do, as they’re doing it in a [virtual] environment,” said Constance Steinkuehler, co-director of Games+Learning+Society.
Davidson isn’t quite so gung-ho about the games’ capacity to serve as standalone, self-contained tests.
“I’m sensitive as a psychologist and a neuroscientist to the complexities of the behavioral and mental processes that are being assessed and nurtured by these games,” he said. “I love the idea of games being an assessment, and I think there’s tremendous possibility there. But I think to restrict the assessment to the game itself is a mistake that would limit the range of information that we could acquire.”
As part of the ongoing study of Crystals and Tenacity, then, Davidson, Steinkuehler, and their respective teams are comparing performance in the games to performance on a separate task in which students are asked to rate the emotions of people in videotaped recordings. The results will be compared against the results of two control groups, each of which are playing comparable commercial games.
Davidson and his team are also looking inside the brains of the research subjects, using a state-of-the-art MRI scanner to determine if playing Crystals and Tenacity is literally rewiring students’ brains to help them be more empathetic and pay attention better.
“We know from other research that game playing as short as 1.5 to two hours can actually produce a change in the physical structure of the brain,” Davidson said. “In our case, we’re asking how does two weeks of game playing...alter brain circuits, in terms of their function as well as their structure.”
Reza Farajian, the lead scientist on the investigation of Crystals and Tenacity, described it this way: If the brain were a city, the researchers would be looking to see if new buildings are getting built, if businesses in key neighborhoods are growing and using more energy, and if the roads connecting those businesses to the rest of the world are being widened and improved.
“It’s totally state-of-the-art stuff,” Farajian said.
The ‘Wild, Wild West’
Gregory Cizek, a professor of educational measurement and evaluation at the University of North Carolina at Chapel Hill, said researchers and educators face a “long learning curve” before they can assess noncognitive skills as well as they currently measure cognitive learning.
“We throw around words like ‘persistence’ or ‘empathy’ or ‘resilience,’” Cizek said. “Those terms have almost universally positive variances, but conceptually, what do they mean? You need a crisp definition to teach it, to build it into a game, to assess it, and to compare students on that characteristic.”
Equally important, Cizek said, is what’s known in assessment circles as “generalizability of transfer.”
“A kid can look at these stringy alien things and maybe demonstrate empathy as defined in a gaming situation, but does that translate to treating human beings differently?” he asked.
Davidson and Steinkuehler readily acknowledge the concerns.
When it comes to designing and studying games to develop and assess noncognitive skills, said Steinkuehler, “it’s the wild, wild West.”
“There’s no other game like [Crystals] on the market,” she said. “We have no other examples of what we’re trying to design.”
Despite the questions, though, Cizek described the idea of using digital technology to assess noncognitive skills as a “natural evolution,” and he described the rigorous ways in which the Wisconsin researchers are delving into the impact of games like Crystals and Tenacity as “awesome.”
And Culatta, of the federal Office of Educational Technology, said he’s excited about the possibilities.
“Nobody gets really excited about going and taking a test, but people do get really excited about playing good games,” he said.