The goal seemed simple: Enter a Watertown supermarket, pick up a few specific things, and check out. Yet just finding the right item can be a significant challenge for Carroll Center for the Blind technology director Brian Charlson and his four young students, all of whom are significantly visually impaired.
“I might want bok choy, but to the bag boy, cabbage is cabbage, and I won’t know it’s wrong until I get home,” Charlson said. “You end up playing a little Russian roulette.”
The group had an advantage, though: a handful of mobile applications designed to help visually impaired people navigate the sighted world. Last week, the students, ages 14-21, took part in the Newton-based center’s weeklong “app camp,” learning how to use apps and accessibility features for the blind on Apple Inc.’s iPhone and iPad.
Developers and advocates say the emergence of hundreds of apps designed for the blind, largely controlled by voice or touch, has made smartphones an invaluable tool for navigating streets and public transportation, checking the value of paper currency, or finding the right product in a store.
“It seems like everybody who’s blind and hip to the tech world now carries an iPhone,” said Ken Nakata, director of accessibility consulting at HiSoftware, a provider of Web compliance products in Nashua.
In addition to specialized apps, the iPhone and iPad offer a feature called VoiceOver, which replaces the familiar swipe-and-tap interface with a system that responds to touch and voice commands with spoken responses. Similar apps and features are also increasingly available on other mobile devices, analysts said, including smartphones and tablets that run either Google Inc.’s Android operating system or Microsoft Windows.
Charlson and the students tested out several apps in a visit to a Stop & Shop supermarket in Watertown on Thursday, using the software tools to help collect and pay for graham crackers, marshmallows, and chocolate — ingredients for s’mores.
The group used an app called Digit-Eyes to identify products by scanning product bar codes and querying a database. Another, called VizWiz, allowed them to take a picture of an item and pose questions about it to an online community of sighted people.
At one point, Charlson posted the question, “What does this box say?” Seconds later, a response was read aloud by the device: “Honey Maid graham crackers,” the phone said.
With VizWiz, “I have 10,000 people in my pocket waiting to help me,” Charlson said. “It’s amazing.”
Once outside the store, Charlson showed the students how to count their change using Money Reader, an app that reads aloud the value of any bill waved in front of the phone’s camera.
While such apps can enable increased independence, they are not a panacea, Charlson said. The real world is messy, and the devices can’t always keep up.
At one point during the Thursday excursion, an announcement over the store’s intercom interrupted Charlson’s attempt to speak a voice command, garbling the result. And while there’s an app that can speak the text on supermarket aisle signs, the chocolate bars and wooden sticks for roasting marshmallows had been relocated to a “seasonal” section, where there were no marshmallows. “This makes no sense!” one frustrated student exclaimed, after finally locating the product three rows away, near the candy. “How would you know?”
VizWiz and Digit-Eyes require aiming the phone’s camera at the right spot, which can be difficult for a person without sight. Other apps can falter when devices lose their data or GPS connections. So even when they have the apps, many visually impaired people will bring a sighted friend, schedule a time when the store can spare an employee to help, or use online delivery services like Stop & Shop’s Peapod for groceries.
Despite their limitations, the apps are still a means to reducing such dependence on others, according to Charlson. “There’s absolutely a rush,” he said. “There’s a feeling of acceptance and inclusion that only comes as a result of a high level of independence.”
Nemanja Djurdjevic, a 21-year-old student at the University of Massachusetts Dartmouth who participated in the Carroll app camp, said that getting an iPhone revolutionized his social life.
“I would always hear kids say, ‘Hey, we’re gonna go to a party. I’ll text you where it’s going to be,’ and I had to be the kid to say, ‘I can’t do that,’” Djurdjevic said. “Now I get to use the same thing as others at the same time, just as effectively. It puts you out there into the world and keeps you connected.”