Data Matters: The Future of EdTech Depends on Sharing Information

If the investment in digital technology and gaming in schools is going to continue to grow, it is up to game developers and companies to do a better job sharing information about what games work and for what kind of learners. That was the message from the deputy director of the Bill and Melinda Gates Foundation who oversees investment in what the foundation dubs the next generation of K-12 education in the U.S. last week.

Stacey Childress said the coming five years represented a real opportunity for digital content developers, but only if they change the way they currently operate.

The Shift to Digital in Education

She said that schools and states were going to look to reallocate some of the $9 billion spent on all textbooks and teaching tools, moving away from more traditional print resources to more flexible and potentially powerful digital assets. But she said that shift will only continue if those creating the games honestly share data about when their tools work and when they don’t.

“Whether you are a for-profit or a non-profit, if you don’t have good information that is valid that you will make transparent about which learners your product will work for or will not be so great for, I think we are likely to stall out at some point, this boom we have going on in edtech, because why wouldn’t we use the same stuff we have always used,” Childress told attendees at the 10th annual Games for Change festival in New York.

“I think it is on… the edtech community at large to take seriously high quality information, transparency, honesty, and a non-proprietary approach to spreading information like that,” she added.

According to Childress, the Gates Foundation is in the midst of a “multi-million-dollar” effort to reinvent education from the ground up and she sees gaming as playing a potentially major role in that change.

She added that if games were going to take on a central role in education then advocates should establish rigorous standards for judging the effectiveness of these new technologies in shaping student performance.

But that means the market must change some of its current structure and open itself up more, something she admitted will not be easy for the industry to accept.

“There are tons of incentives in the market for information to stay locked up inside proprietary silos, for big players that have enormous long-term contracts with states and large districts to keep that information as close to the vest as possible. Look, they’re not bad people, the market rewards that right now,” she said.

Assessing What Is Known about Games and Learning

As part of this effort to make information about the industry more transparent, Childress announced the publication of two new research reports focused on assessing what is known and not known about gaming and learning.

Childress said the Bill and Melinda Gates Foundation sponsored a “meta-analysis” of all the research done between 2000 and 2012 that touched on games and learning. Of the nearly 60,000 pieces of research that fit the bill, she said only about 700 of those undertook some experimental effort to formally test the effects and only 77 of the research papers had a methodology that either had a control group or a pre- and post-test to clearly tested the effects of games on the student.

The report, which will be released in greater detail in the coming months, stressed the importance of how games are designed when it comes to how effective they can be for students.

“Much of the debate about digital games for learning to date has focused on more simple questions about whether games are good or bad for learning,” the report’s authors conclude in their phase one summary. “The findings of this meta-analysis demonstrate that the efficacy of digital games for learning depends on their design.”

Childress said although the research raised many questions for future study, it did find that students in the control group could have seen their scores increase 12 percent if they had been exposed to learning games during their lessons.

That news brought applause from the crowd of serious game developers gathered in New York for the three-day conference.

“Isn’t it nice to know that,” Childress said. “We’ve all been saying this stuff like that for a while” but now it can be statistically backed up. The report, from the independent research institute SRI, found that none of the studies demonstrated that inclusion of games hurt learning outcomes. All students did better to some extent.

But the research also highlighted surprising gaps in what is known about the effects of games and what is sometimes widely claimed.

Childress said the research has largely focused on students’ ability to develop cognitive skills and remember content, an area she said is often not the focus of claims about what games can help teach.

SRI’s researchers found only 1 percent of the experimental research done on games had tested whether they could help students develop skills like communication, problem-solving and working in teams – what the SRI team called “interpersonal competencies.” The summary concludes that games’ impact on this area “are less clear due to low numbers of studies focusing on these outcomes.”

Childress said advocates have often stressed those interpersonal skills when talking about the power of gaming, but that those advocates should be cautious until this gap in research is filled by credible studies.

“We have got to get serious research going with the appropriate kind of measures … about these claims we are making about communication, collaboration – these sort of 21st century skills stuff. It’s a great hypothesis. It seems like it would hold. We do not, at this point, because of the tiny number of studies that reach a high enough research bar, actually have the evidence to make those claims,” she said.

She added that it was these same interpersonal skills that are the key to future success and if gaming is to be seriously considered part of the educational method to teach these inter- and intrapersonal skills, they must be studied.

If gaming is to play a more central role in education in the future, it must be able to back up its claims with solid research, saying in education, “The bar for what passes for legit is pretty darn high and … it should be high.”

But the research did provide evidence that gaming is effective in helping students with so-called cognitive learning, the traditional recalling of facts and theories, a result that Childress said should hearten those working on educational games.

She stressed that the foundation continues to see games as a natural tool for education, that their very structure makes them ideal, saying, “The fact that a game has, as one of its key components, a set of rules for engagement that a learner has to sort of figure out how to follow; it has goals that a learner has to reach to keep moving through the game; and then a way of capturing and displaying a learner’s progress through those. I mean, those are just great components of any good learning environment, so the fact that they are a natural part of any good games design makes them an interesting part of a digital tools environment.”

The key now, she said, is to help teachers, parents and others more easily unlock that power for students.

 

Related:

• Research on Assessment in Games (GlassLab-Research), SRI International
• “Can Digital Games Boost Students’ Test Scores?” by Tina Barseghian, MindShift

Lee Banville is the editorial director of the soon-to-launch gamesandlearning.org, a project of the Games and Learning Publishing Council. Watch for more in-depth coverage of the games and learning field here, and on gamesandlearning,org, launching in Fall 2013.