How emotional AI can benefit businesses

Emotion is one of the most distinguishable human qualities, one that sets us apart from machines. However, it is not out of the realm of possibility for machines to read emotions and respond accordingly. Increasingly, machines are able to interpret human’s emotional states and adapt their behavior to give appropriate responses—something we call emotional AI, or artificial emotional intelligence (though in the computing field, it is known as affective computing).

Here we will explore what it is, how it works, and how it can benefit businesses.

Three types of emotional AI

Emotional AI is the next step in the evolution of artificial intelligence. By interpreting people’s emotions, AI can respond in a much more naturalistic manner, making the interaction much closer to typical human intercourse.

There are three main types of emotional AI: natural language text analysis, voice analysis and facial expression analysis. The first two are already quite common, while the third probably attracts the most media attention. Other types of analysis also include mouse movement, eye-gaze, heart rate and electrocardiography, etc.

● Natural language text analysis

It involves AI scanning written text like a review of a product or service, online articles or tweets, and then picking up on the sentiment of whether it is positive, negative or neutral.

● Voice analysis

This analyses a user’s speech signals like their vocal pitch, intonation and tone as well as the words they use to determine their sentiment. For example, someone with a dry sense of humor might say the opposite of what they actually mean for comic effect, but using voice analysis you could pick up on the true meaning of what they are saying. This is especially useful in call centers—detect an angry tone of voice from a caller, and you can transfer them to a human operator rather than risk frustrating them further by making them deal with an automated system.

● Facial expression analysis

This is perhaps the most interesting one. Using a video camera to read someone’s facial expressions, AI can analyse their emotions, and from that you can infer their state of mind, their intentions, whether they are lying or being genuine, and so on. Some startups already use this in their job interviews to determine whether the interviewee is nervous, confident, or sincere about his answer, etc. It also has enormous potential for financial services companies, such as banks or fintech firms, when they are deciding whether to approve a loan for someone.

Analysing emotions for retail: online and in-store

Emotional AI will be useful to all kinds of businesses. If your company needs to understand human emotion in order to make a decision, you will have a use for emotional AI, as it can help automate this analysis and hence these decisions. Nowhere is this truer than in the retail space.

Using cameras in stores, emotional AI can observe your customers’ facial expressions, how they walk, and other variables that will help determine their emotional state. For example, if someone is frowning a lot and walking at a very fast pace, you could deduce they are stressed and in a hurry. In which case, you could advise your salespeople not to approach them to tell them about your latest offers.

We can do the same thing online. Instead of seeing a shopper’s body language you can analyse their online behavior patterns. If they use their mouse cursor very aggressively, for instance, AI can infer they are stressed and/or in a hurry, and less open to offers. On the other hand, if they hover the cursor over the ‘Buy now’ button for a while, they might be indecisive, and so it could be a good time to send them a coupon for a discount or free shipping to help convert them into a paying customer.

Nailing an emotion: accuracy and interpretation

So how accurate is emotional AI?

In the case of facial expression analysis, researchers have defined about 64 different facial expressions and micro expressions. AI can detect these with a pretty high degree of accuracy, and this will only improve as the technology develops – like other forms of AI, emotional AI improves as more data you feed it. So, accuracy isn’t the issue.

Problems arise when you get onto the question of interpretation. Just because someone pulls a certain facial expression in response to a certain question, that doesn’t necessarily mean they are lying. It is a big leap to go from recognising a facial movement usually associated with one kind of emotion or behavior, to assigning a certain motive to that expression. People’s emotions can be quite unique, and different people show emotions differently – there is a huge amount of variation in how people react to certain situations. You have to go person by person, and be wary of devising universal rules that you apply to everyone the same.

In demos, the facial expression recognition feature of emotional AI looks very impressive. That is because in those demos, researchers asked actors to pretend to be nervous, and AI picked up on that emotion. However, actors tend to over-emote—in the real world, people behave very differently. In real scenarios, I would say it is accurate 70% of the time, but use an actor and that rises to about 90%.

Coupons yes, job denials no: Putting it to good use

You probably shouldn’t use a system with 70% accuracy to make “final” decisions like whether to hire someone or deny them a loan. That is because you are taking the final decision out of their hands, which is a lot of responsibility for a company to wield. However, pairing humans and AI to make an improved final decision can already be useful in certain scenarios.

For instance, emotional AI is very well suited to marketing decisions like whether to send someone a coupon. With this kind of decision, the final say is still in the customers’ hands, rather than the company’s. You as a marketer are just nudging the customer to complete the purchase.

Marketers should start preparing for emotional AI by focusing on data that reflects your customers’ emotions. For example, a customer call center can record all calls, and a website can store all user reviews for analysis. Use this data, and soon you will be able to leverage this technology to make more effective business decisions.

It is still relatively early days for deploying emotional AI in the real world, but it is a crucial step in the development of AI, and absolutely essential if we want to build an AI that interacts naturally with humans.

Dr. Min Sun is the chief AI scientist at Appier.