Fast food and fast fashion have something in common – aside from speed. For more than 90 years, KFC has been serving Colonel Harland Sanders’ original secret blend of 11 herbs and spices to chicken lovers everywhere. But in 2016, KFC China added a new, unexpected ingredient to the late Colonel’s famed recipe – artificial emotional intelligence or ‘emotion AI’.

Strategically partnering with search engine giant Baidu – the so-called ‘Google of China’ – KFC opened the doors to its first smart restaurant deep in the heart of Beijing’s financial district.

The emotion AI-enabled restaurant used image recognition technology to scan customers’ faces and recommend menu items based not only on their estimated age and gender, but their mood too.

Similarly, a year earlier and some 9,000 kilometers away at Pitt Street Mall in Sydney, Japanese clothing brand UNIQLO rolled out its emotion AI-powered UMood kiosks.

Delivering a unique in-store experience, UMood uses wearable technology to carry out a brain wave analysis on participating customers, which identified their moods and recommended an ideal T-shirt, from a range of more than 600.

In both examples, emotion AI assumed the role of mind reader by anticipating what food and fashion items customers wanted to buy.

What is Emotion AI?

Emotions drive behavior, so it stands to reason that a retailer’s dream is to be able to get inside the hearts and minds of consumers. Emotion AI is a branch of artificial intelligence that enables measurement, understanding, simulation and response to human emotion.

Essentially, a retail bot with emotion AI capabilities can read customers’ feelings through text, speech and facial expressions. It can then interpret their emotions and, most importantly, adjust its reaction accordingly.

Emotion detection and recognition technology combines behavioral and sensory data, which empowers retailers to hyper-personalize the customer experience and help them achieve the lofty goal of real-time empathetic marketing. In other words, it’s reading customers’ emotions and then adapting their experience in real time.

Our Expert Roundtable

To discuss the potential impact of emotion AI on the retail sector, The CEO Magazine convened a roundtable of AI experts.

Rodrigo Madanes, Global AI Leader at EY

Rodrigo Madanes believes that “emotion AI might soon become a new instrument in the retailer’s toolkit”. Referring to the prevalence of cameras in ordering kiosks, he says it may not be long before emotional data is used to “power up” marketing engines.

“When you talk with a salesperson, much is communicated beyond saying ‘I like it’ or ‘I don’t like it’.

“With emotion AI capabilities, technology will soon be able to learn more about customers’ emotional states, so they can have more personalized interactions and receive more personalized offers.”

Despite Madanes’ optimism about the promise of emotion AI technology, he cautions that it should be introduced carefully.

“A focus on responsible AI can’t be overstated here,” he says. “The data used to detect emotions is very rich, so one needs to respect regulations and privacy concerns. Also, cultures differ, so a one-size-fits-all approach isn’t going to work well.”

Annette Zimmermann, Vice President Analyst at Gartner

Annette Zimmermann recently published a research note discussing trends and controversies surrounding emotion AI.

“The majority of emotion AI vendors are focused on primary research, customer profiling, prototype testing and other consumer-oriented use cases where the technology is used for the sole purpose of observing an emotional reaction towards a product or service and subsequently making improvements,” she points out.

“Another group of vendors uses emotion AI to provide emotional analysis for contact center conversations. Again, this is to improve customer service.”

Zimmermann echoes Madanes’ concern about potential privacy issues, particularly in a retail environment.

“The problem with emotion AI is its privacy concerns,” she reflects. “How do you communicate to customers that in-store cameras are detecting their emotions?”

Zimmermann also observes that emotion AI hasn’t experienced widespread adoption in retail. “Emotion AI technology has been tested in retail here and there, but it hasn’t caught on yet,” she admits.

“Even though it can be applied to a retail setting, I don’t see many vendors focusing on this use case – except in China.

“But recently, a Singapore-based vendor told me they’re currently testing emotion AI in an office setting where cameras detect the overall mood.”

Robert Hetu, VP Analyst at Gartner

Robert Hetu highlights retailer inefficiency as a weakness that emotion AI technology can potentially minimize or even eliminate altogether.

“Retailers aren’t very good at understanding customers, nor are they good at interpreting direct feedback from customers into strategies, such as store design or path-to-purchase design,” he says candidly.

“By leveraging technologies, such as computer vision or voice analysis, more context can be applied to interactions to better understand what the customer is thinking and feeling in the moment.

“Some retailers, for example, are using computer vision and AI to monitor self-checkouts. In this case, the technology isn’t only used to detect theft, but also identify that a customer may be struggling with the technology.

“There’s also the possibility of identifying frustration or curiosity in a store that could lead to direct communication in the moment to encourage a purchase or send a store associate to assist a customer trying to make a purchase decision.”

The Future of Emotion AI

On balance, real-time empathetic marketing holds great potential for retailers. Being able to read customers’ emotions and adapt the customer experience based on their real-time reactions equips retailers with the right message, the right product and the right person at the right time.

But emotion AI technology does have its limitations, both in principle and in practice. While it undoubtedly takes customer-centricity to the next level, the challenge for retailers is to take customers’ privacy concerns seriously and adopt robust security protocols to protect their personal data – even if it’s only used in helping them choose a burger or a T-shirt.