Senior software engineer David Alfonso of Boston-based Pison Technology doesn’t resemble the sorceror’s apprentice from the old Walt Disney cartoon “Fantasia.” But with a wave of his hand, he seemed almost as powerful.
Armed with only a high-tech cuff wrapped around his wrist, Alfonso pointed an index finger at the robotic guinea pig Pison is using for its experiments, the four-legged machine from Boston Dynamics named Spot. He raised his hand toward the ceiling, and the machine rose to its feet, and then marched across the floor as Alfonso pointed across the room. Then he swept his hand downward, pointing at the floor, and the robot dutifully reversed its steps and curled up at his feet.
Advertisement
Alfonso was demonstrating Pison’s new gesture-control system, a technology that has so far eluded corporate giants such as Microsoft that have spent hundreds of millions trying to perfect it. Pison, a startup spun out of the Massachusetts Institute of Technology, says it has developed a practical way to control all sorts of digital devices by intercepting the electronic traffic between our hands and our brains, and translating them into commands the machines can understand.
The system was invented to help people with the nerve disease amyotrophic lateral sclerosis, also known as ALS or Lou Gehrig’s disease, and others who have limited or no control over their muscles. Just the thought of moving their hands is enough.
“We allow touchless control, regardless of capability of a human body,” said Pison’s founder, Dexter Ang.
Earlier gesture-control systems, such as Microsoft’s ill-fated Kinect gaming device, used a boxful of sensors to track the user’s motions. But the Pison wrist-mounted sensor resembles a smartwatch. It detects “biopotentials,” the nerve signals being sent to a user’s hand from his brain. The device also contains chips that track hand and arm motions. All this data is sent to the user’s smartphone, which can issue commands to a computer, a robot, a flying drone, or a living room light switch.
Advertisement
And unlike Microsoft Kinect, Pison’s technology is portable and can be used anywhere.
The doglike Spot robot that Alfonso used is designed to carry multiple drones on its back. A demonstration video shows a rescue team steering the Spot robot through a forest by pointing the way. Then they use gestures to launch several drones which use video cameras to search for an injured hiker.
“This add-on technology will be particularly useful for when multiple Spots are being deployed and in specific applications like search and rescue,” Boston Dynamics vice president Michael Perry said in an e-mail.
Pison is also working with Microsoft, which is looking to add gesture control to its HoloLens augmented reality system. An industrial worker or soldier wearing an AR headset sees digital images overlaid on the real world. With a Pison system, he could interact with nearby machines by gesturing, with no need for touchscreens or joysticks.
Pison plans to have a commercial version of its system on sale late this year or by early 2022. The company foresees plenty of consumer applications: A smartphone user could play his favorite podcast by twitching one finger, or call up a weather report by wiggling a thumb. Ang foresees almost limitless uses.
But he launched the company for a more personal reason. In 2014, Ang’s mother, Chiu Ang, was diagnosed with ALS, an incurable disease that left her unable to control her muscles.
Advertisement
“She couldn’t pick up a spoon to feed herself,” Ang said. “She couldn’t move her hand to click a mouse, and she couldn’t do her favorite activity, which was to read books.”
Ang quit his job at a futures trading firm in Chicago and moved back to Boston to care for Chiu. At one point, she asked him whether someone could invent a way to detect the signals from her brain to her hand, so that she could once again control an e-book reader. Although Chiu Ang died in 2015, the suggestion stayed with her son. He returned to MIT, joined a non-degree advanced studies program, and earned a $25,000 grant from the school’s Sandbox Innovation Fund Program to get Pison off the ground. The company has since raised about $5 million, much of it coming from the venture capital arm of audio equipment maker Bose Corp..
Today, Pison is carrying out clinical trials with about 20 people with ALS, including Steve Saling, a landscape architect who was diagnosed with the disease in 2006. Today, Saling cannot talk and can barely move. But he can type messages using a system that recognizes the motions of his head. And he’s working with Pison on an upgrade.
“The project that we have been developing together is a smart switch that can be used as a clicker on my computer or be used as a mode or power switch on my wheelchair,” Saling wrote. “Pison has been able to tap into my nervous system and interpret the signals that are generated by just thinking about movement. It looks like magic but it is just science.”
Advertisement
Douglas Weber, a professor of mechanical engineering and neuroscience at Carnegie Mellon University, said the technology has potential for people with disabilities.
“It’s wearable, it’s low-cost, you can build them with off-the-shelf components,” he said.
But Weber noted that Pison faces tough competition. In 2019, Facebook paid $1 billion for CTRL-Labs, a New York company developing a similar system. It’s believed that Facebook wants to add gesture control to its Oculus virtual reality videogame system, as a substitute for clumsy handheld controllers.
Weber also doubts that gesture control will be embraced by most consumers. “Keyboard and mouse work pretty well for most things,” he said.
But Ang believes there’s a limitless market for Pison technology.
“We are only at the beginning of this new paradigm,” he said. “We imagine that our technology can be applicable to everybody, long-term.”
Hiawatha Bray can be reached at hiawatha.bray@globe.com. Follow him on Twitter @GlobeTechLab.