CMU Technology Syncs Smart Watch User’s Actions With Helpful Apps

Nov 23, 2015

Imagine turning on your motorcycle and then the smart watch you're wearing automatically vocalizes the estimated time to return home based on current traffic patterns.

Carnegie Mellon University teamed with Disney Research to create technology for smart watches that can do just that -- detect what the user is holding to offer related information.

But it doesn't work with anything, just electronic or electromechanical objects, according Gierad Laput, lead researcher of the “EMSense: Recognizing Handled, Uninstrumented, Electro-Mechanical Objects Using Software-Defined Radio” project at CMU. When your skin touches an electronic object, your body acts as antennae for those signals.

“The sensor that we built can sense these signals that flow into your body and then we use those to detect the objects,” Laput said.

Although detection is limited to those electronic or electromechanical objects, smart watches will be able to give real-time identification of what the user is holding. Passive or nonconductive objects, such as plastic cups, silverware and pens, will not be recognized because they don’t give off EM noise.

This ability to differentiate between objects will offer the opportunity for “context-aware” smart watch applications.

video from Disney Research shows how this new feature could enhance everyday life. The clip shows a user brushing his teeth. As he brushes, the watch recognizes that he’s holding the toothbrush, which causes the watch to start a 60-second timer. Then as the user goes to make breakfast the watch detects him opening the fridge and turning on the stove. The watch then asks if the user would like to listen to the morning news report.

The new EM-sense technology could also make office life a little easier.

“[The user] touches his office door and then it tells [him] 'Hey, you have an appointment in five minutes.' And then the user says 'Okay remind me to pick up the milk on the way home.' And so at the end of the day, the user closes his door and then the system says, Don’t forget to pick up the milk,'” said Laput.

Other uses for the context-aware smart watch apps include guiding a user through a project with step-by-step instructions based on the power tool being used. It could also unlock a laptop without a password when synched together by the owner. 

According to Laput, the team also programmed “background subtraction,” which cancels out any additional EM noise (possibly from fluorescent lights or radio broadcasts) , except from the object being touched.

Laput speculated that the sensing technique will be easy to integrate into existing smart watches, but will need the addition of an EM sensor first. He said companies like Apple or Disney have the resources now to fit the technology into existing models.  

“This is one of those applications that just opens up new ways on how you can use and make use of smart watches beyond what they’re being used for right now, which is just notifications and pedometers and heart rate,” he said.