fb-pixel Skip to main content

Feeling emotional? The machines know

A Global Customer Solutions Specialist at the Met Life call center in Warwick, R.I. Jonathan Wiggs/Globe Staff/Globe Staff

A call center worker sees a heart icon pop up on her screen, a signal to her that the customer on the other end of the phone is getting emotional.

An analysis of a job applicant’s video interview compares his body language and facial expressions with those of previous successful hires to help determine if he will be a good fit.

Doctors use artificial intelligence to diagnose patients; sales reps use it to learn more about their customers — and get them to buy more things.

Across sectors, AI is transforming the way we work, with programs that can analyze thousands of data points instantaneously, and, ideally, help us do our jobs, or even help us get one.


In Massachusetts — where there are more machine-learning and AI-related job openings per capita than any state except Washington, according to the Boston-based job market analytics company Burning Glass Technologies — companies have developed programs that gather information on customers while they’re on the phone, chatbots that can answer human resources questions, and platforms that can predict our e-mail patterns.

Last fall, the Massachusetts Institute of Technology announced it was investing $1 billion in a new college dedicated to computing and artificial intelligence, and there’s sure to be a huge demand for its graduates. According to a global survey by the Connecticut research and advisory firm Gartner, more than a third of organizations use AI, a 270 percent increase compared with four years ago.

Thus far, AI has been displacing workers, according to Gartner, but by 2025, AI is expected to create 2 million net new jobs. Many applications, like the one developed by the MIT Media Lab spinoff Cogito, are intended to work alongside us.

Emily Baker is a supervisior at the Met Life call center in Warwick, R.I. Jonathan Wiggs/Globe Staff/Globe Staff

Cogito can detect emotions and speech patterns in phone conversations and then give customer service and sales agents suggestions for improving a call. A coffee cup icon, for instance, is a signal to speak with more enthusiasm, as low energy interactions can be seen as a sign of disrespect or a lack of interest, said chief behavioral officer Skyler Place. After the insurance giant MetLife started using Cogito at its call center in Warwick, R.I., last year, supervisors realized many of their agents were fast talkers when a speedometer icon repeatedly urged them to slow down.


Human instincts are still vital, of course. When a caller to the Warwick center joyfully announced, “I’m 86!,” a heart icon — it’s pink, of course — appeared indicating an “elevated emotional state” detected by a tension in the vocal chords and changes in pitch, tone, and pace. The heart prods workers to be empathetic, and the agent was momentarily thrown, wondering, “Do I say, ‘I’m sorry’?” But, because he could tell the customer was happy, he congratulated her instead.

It’s all about helping agents become more engaged with customers, who are often in distress after losing a loved one or going through a car accident or house fire, said Kristine Poznanski, executive vice president of global customer solutions for MetLife.

“You don’t think about AI as enhancing a human interaction, but that’s exactly what this is doing,” she said.

There was concern at first that agents would see the program as Big Brother, she noted, but because agents are able to resolve calls more efficiently, their performance is improving, and with that comes the opportunity for higher pay and a better schedule.


Plus, as Cogito’s Place puts it: “They’re not getting yelled at as much.”

Companies are also using AI to learn more about their customers’ habits and spending power and to determine who is at risk of canceling a service.

Using AI-powered software developed by Pegasystems in Cambridge — whose customers include Sprint and General Motors — a cellphone provider can track customers who are looking at termination clauses on the company’s website or posting angry tweets about its service and can then reach out to them proactively to offer an upgrade or discount, said Rob Walker, Pegasystems vice president of decisioning and analytics.

If it’s a particularly valuable customer — one who racks up a lot of roaming charges, for instance — the system may urge the sales rep to sweeten the deal accordingly.

Or a bank could use the software to determine which of its customers is a good candidate for a home equity loan — say, someone with a decent income who has started shopping at home improvement stores — and send a text offering a line of credit immediately after the customer uses his bank-issued credit card at Home Depot.

Yet, along with the privacy issues raised by monitoring people’s purchasing patterns and Internet activity to drum up new business, a system that consistently rewards high-dollar customers could promote a bias against lower-income customers, warned Darrell West, founding director of the Center for Technology Innovation at the Brookings Institution.


AI is “being deployed very broadly, so we really do need to understand the societal and ethical consequences so that we end up with a system that serves human needs,” he said. “There’s a lot of concern that humans are no longer in charge of the technology and that algorithms will make all the decisions.”

But even if humans are totally in charge, the programs they create inevitably reflect, and even amplify, existing biases. Amazon learned this after discovering that its AI-powered recruiting system preferred male candidates because its data were based on an analysis of 10 years of resumes submitted to the company that came largely from men, according to a Reuters report.

The recruiting industry in particular has started turning to artificial intelligence in recent years. HireVue, a video interview service based in Utah, offers a program that predicts job candidates’ potential success by using video interviews to compare their facial expressions, vocabulary, tone of voice, and body language to top performers in similar roles at the same company. Of the 800 corporations that use HireVue’s video interview services, about 120 are set up for AI-enhanced video comparisons.

Because AI is still “a little voodoo,” some companies might just be experimenting with it, said Kyle Lagunas, a research manager at IDC, a market intelligence firm in Framingham. But the potential cost savings and improved efficiency make it difficult to ignore.

Scout Exchange, a Boston recruiting marketplace and software developer that analyzes recruiters’ track records and predicts which jobs they are best suited to find workers for, said companies using its platform are filling jobs 20 to 50 percent faster than before.


“You have to kiss a lot of frogs to get to the prince, so you want to reduce the number of frogs you kiss,” said chief executive Ken Lazarus.

Separating the wheat from the chaff is also the goal of an AI platform that can pick out the most urgent messages in your inbox and predict how much time you’ll need to address them. Started by a group of executives seeking a solution to information overload, Knowmail, based in Woburn, studies past e-mail correspondence to learn how an individual uses his or her inbox: Which topics and correspondents spur you to respond quickly? Is the message related to an upcoming meeting? Is it from your CEO?

Within a few minutes of installation, Knowmail — which is integrated into Microsoft’s voice-activated assistant Cortana — can predict and prioritize your actions, providing an augmented view of your inbox that highlights urgent messages, as well as important ones that can wait.

“Think of it as kind of a mini-me inside a computer,” said chief executive Haim Senior, noting that the program can run strictly on a device, instead of using the cloud, if privacy is a concern. “You look at your inbox the same way you looked at it 10 years ago, 20 years ago, and it’s just not enough anymore.”

Katie Johnston can be reached at katie.johnston@globe.com.