New Hybrid Digital-Physical Games May Improve Student Learning

May 9, 2017

Nesra Yannier said, growing up in Turkey, school was kind of boring.

“The education system was based on memorization, so I always thought it should be different and should be helping kids understand the reasons rather than memorizing facts,” she said.

Nesra Yannier developed NoRilla as part of her Ph.D. program at Carnegie Mellon University.
Credit Carnegie Mellon University

When Yannier was working on her Ph.D. at Carnegie Mellon University, she sought ways to make learning more engaging and struck upon the idea of pairing digital applications with real-word educational toys.

Along with her advisors ​Ken Koedinger and Scott Hudson, Yannier developed a game called NoRilla, which teaches physics at an elementary school level. The digital app is hosted by a gorilla character, who instructs children to place two different towers on a table.

“He asks users to make a prediction about what’s going to happen … about which tower will fall first,” Yannier said.

Kids have to consider how tall each tower is, how big the base is and whether it’s top heavy. Then they push a button and the table begins to shake.

Children play with NoRilla at Ikea during a pilot project for the game.
Credit Carnegie Mellon University

Students can also build their own towers or complete challenges, like building a taller tower or one with a small base that will stay upright on the shaking table.

Yannier said her research with more than 200 children showed at the NoRilla system improves children’s learning five-fold, compared to equivalent digital-only games. With funding from CMU, the National Science Foundation and the Academy of Television Arts & Sciences, she’s now working on turning her research project into a commercial product as part of the AlphaLab Gear accelerator program.

“Everybody learns best when brains are fully engaged and being hands on and being tactile is a good way to engage in another dimension of the brain,” said Jesse Schell, CEO at Schell Games on the South Side.

The company sells a game called Happy Atoms, which marries a traditional molecular modeling set with a Pokemon Go-style app.

Happy Atoms challenges children to build molecules in the real world and complete quests in the digital world.
Credit Liz Reid / 90.5 WESA

“Instead of can you catch all the Pokemon, it’s can you build all the molecules,” Schell said.

Schell said these kinds of hybrid physical-digital toys aren’t a fad, they’re the beginning of a long-term trend. He said that’s particularly true when it comes to the “internet of things,” which refers to the embedding of internet connectivity in everyday objects, like a thermostat or a toy.

“Toys and games usually are first in terms of technology because they can afford to be imperfect,” Schell said. “And so if you care about the idea of the internet of things, you should watch the toy market closely because that's where all the innovations are going to show up first.” 

In this week's Tech Headlines:

  • Pittsburgh-based Michael Baker International has landed a contract with Hartsfield-Jackson Atlanta International Airport to use drones to inspect runway conditions. The one-time contract is a first of its kind contract according to the company. The hope is to provide daily inspections in the future. The process would collect large data sets, which can be analyzed for changes and used for maintenance planning.
  • A Burger King ad has revealed the hackability of voice assistant technology. The fast food chain's 15-second TV ad included an announcer saying "OK, Google" and asking a question about the Whopper. In many homes, the device obediently began reading the burger's ingredients effectively extending the commercial.  "It's a wakeup call," said Earl Perkins, a digital security analyst at the research firm Gartner. "It's a harbinger of things to come." The idea may have been planted accidently by Google itself. A Google ad during the Super Bowl that used the phrase "OK, Google" reportedly set off people's devices. Many experts believe that deliberate attempts to trigger voice assistants will be short-lived. Among other things, brands have to face the consequences of potentially annoying millions of people. Developers say they have created techniques to block TV ads from activating their devices but have not provided details.

The Associated Press contributed to this story.