Tech companies are developing artificial intelligence programs to read our emotions. Actually, they’ve already developed such programs and they are currently trying to ‘perfect’ them. They are designed to tell those who sell to us, employ us and police us what we are feeling. Sound sinister? I’m not a conspiracy theorist (I don’t look good in tin foil!), but there is undoubtedly something unnerving about this sort of technological advance. Every time I look closely at a sidebar on a website I’m astounded by how targeted the advertising is. It is obviously informed by my web history and demographic profile (I can’t be alone in feeling deflated when I notice how many funeral plans or retirement homes pop up… let alone some of the more intimate recommendations.) So, even for someone with a modest social media footprint, there is already plenty of data for organisations to get their teeth into; but tracking emotions? That’s really getting personal. The ethical ramifications of technology like this are immense. What makes this whole area of techno-progress even more worrying, though, is that the science upon which it is based is, at best, inconclusive.
There is an assumption built into the development of emotion-sensitive AI systems that humans all express emotions the same way. Personally, having sat in a Counselling room opposite clients who laughed as they told me about the collapse of their relationships or cried whilst they described their child’s first steps, I have my doubts. Are emotional responses universal? Is the way we express them common to all cultures? After 150 years, the jury is still out.
The case for the defence
In 1872, Charles Darwin published The Expression of the Emotions in Man and Animals. He argued that all humans show emotions - such as anger, disgust, fear, surprise, sadness and happiness - through strikingly similar behaviours. Darwin also believed he could identify common emotions and their expression in ‘lower’ animals. Although not particularly popular at the time (after all, it not only harked back to the troublesome ‘evolution’ debate, but implied common humanity across ‘races’) this became the orthodoxy. The work was illustrated with photographs of people pulling faces to express emotions such as those mentioned above. Early photographs from his work are - perhaps surprisingly- accessible online... and fascinating.
Building on Darwin’s ideas, researchers have been asking people what emotions they perceive in faces since at least the 1960s. Realising that one barrier to universality may be cultural variation, psychologist Paul Ekman carried out observations and interviews of people from many cultures, including indigenous groups in remote geographical areas. He concluded that emotions were interpreted consistently across cultures, therefore the emotions themselves must be universal.
Today, you can easily find images of faces representing expression of these and other emotions. Indeed, the interpretation of these expressions is used to assess emotional intelligence and empathy. I have personally been involved in training sessions in Commercial, Therapeutic and Educational settings which leaned heavily on this research. As I said, it has become the orthodoxy. It’s not surprising, as there is obviously at least some truth in the idea that your anger can look like your adversary’s, your sadness like mine. If empathy is about sharing someone else’s inner world, then along with what they say and how they behave, what their face tells us of their experience is a crucial part of human social interaction. If people find it difficult to ‘read faces’, we tend to think of them as socially and emotionally disadvantaged.
The case against
The thing is, there is now evidence that interpretations of emotional states vary far more than Darwin or Ekman believed. Ekman and his colleagues always accepted that it was possible to ‘fake’ emotions and that you could suppress the expression of feelings. As a result, there wasn’t always a perfect correlation between what went on inside a person and what they expressed. Many researchers now believe the range of variation is much greater than was first acknowledged, and if the range of variation becomes too great, the idea of universality is somewhat undermined!
What is more, the Canadian Lisa Feldman Barrett and others have recently popularised (and gained academic credibility for) the idea that emotions are not hard-wired. Her ideas about “constructed emotions” are quite sophisticated, so I won’t go into them in depth here - largely because I’m not sure I fully understand!
Essentially Barrett believes that we take in information from the world around us and compare the inputs to distilled models of how we believe the world works. We then respond to the collection of stimuli on the assumption that they represent something that fits the model to which are brains have matched it. Confused? Try this… ‘if it looks like a duck, walks like a duck and quacks like a duck then it's probably a duck.’ I’m not being overly flippant here. Our interpretation of a set of inputs relating to the shape, colour, movement and sound coming out of the object lead our brain to draw the conclusion that the thing in front of us is a duck. This is based on all of our previous experiences of ducks. If we had never come across the concept of a duck before, we might categorise it differently. This happens in an instant. We aren’t aware of the processing that takes place to come up with our duck conclusion. As a species we like to be able to make sense of the world around us… to pattern-seek, to classify and categorise and as individuals we do the same. Most of us are fine with this idea as it relates to external concepts such as duck. Barrett takes the more complex and sometimes controversial step of applying it to our emotional concepts… such as anger, disgust, fear etc.
In her best-selling book ‘How Emotions are made’, Barrett gives the example of her reaction to a tragedy viewed on television news. Her stomach knotted and her face flushed as her heart beat faster. Her response was to cry and relieve some of the intense physical sensations she was experiencing. Rather than assuming that this all arose in response to the event she was watching, however, Barrett proposes that what was actually happening was:
“I learned long ago that “sadness” is something that may occur when certain bodily feelings coincide with terrible loss. Using bits and pieces of past experience, such as my knowledge of shootings and my previous sadness about them, my brain rapidly predicted what my body should do to cope with such tragedy.” And the best response was to cry.
In other words, she believes that witnessing responses to tragedies earlier in her life caused her body to predict the impact of such an event on her when she came across a similar experience. The experience of sadness wasn’t a simple reaction to something happening on the outside. It was the result of complex systems working together to make a self-fulfilling prediction about what was needed for her body to cope… based on what had been needed in the past. In this case crying was an important part of relieving the pressure that came from her other physiological responses.
Barrett’s work suggests that what we sense about our outward environment and our internal body function (called interoception) are actually concepts or simulations not just reactions to reality. We have an idea about what we are going to experience in advance of the event and the way we go on to experience it is a combination of the experience itself and our prediction of what it will be like. The information coming from our senses influences what we perceive; it does not define it.
Even if you’ve followed the last few paragraphs, you may be saying to yourself ‘so what?’
One of the implications of the Theory of Constructed Emotions is that, if you don’t have a concept of an emotion to match an experience against, you won’t be able to perceive the emotion. You will still have the bodily sensations, but your brain won’t be able to pin a precise label on them. If a person is unable to develop a concept of an emotion and identify bodily sensations with this concept, their emotional experience will be limited. If you think about it, this could occur because of a problem with brain function or with the reinforcement of emotional experience as both go towards the construction of the concept of the emotion.*
If you haven’t learned, or been taught, about your emotions, you may not have the concepts available to make sense of them. If you have learned about them in such a way that your responses match to a different emotional concept, then you’ll interpret the same set of information and physiological responses as a different emotion to the one experienced by the person next to you. In other words, emotions are not hard wired, they are made; they are not universal, as well as the way our brain works they depend on things such as childhood experience and the culture in which you are raised.
The once widely accepted principles of universal, hard-wired emotions are now being challenged by credible researchers. Barrett’s ideas aren’t universally accepted and I have heard her talk about the anger her theory has prompted. It seems that for some it is an attack on emotions themselves. She doesn’t see it that way. For her, understanding the origin of emotions doesn’t make them less important or reduce us to a set of behavioural responses.
Whether she is entirely right or entirely wrong or, as is most often the case with theories, partly right, Barrett’s ideas challenge the assumption that emotions are universal. And if emotions aren’t universal, then AI which interprets our emotional responses becomes not just dubious in ethical terms, but invalid and untrustworthy. I don’t mean to sound like a Luddite, but I’m not entirely sure what good would come from such technology if it were proven to be reliable. After all, it amounts to second-guessing people’s emotional states and, one assumes, basing serious decisions on the interpretation of a non-living, non-feeling machine. Now that is reducing people to a set of behavioural responses.
.* For me, this could bear out why those with severe attachment issues can present with having similar difficulties in recognising and decoding emotions as some neurodiverse people on the autistic spectrum. In one case the experience is lacking, in the other the way of processing the experience is compromised. The result is the same, a restriction of the range of emotional concepts the individual has developed, so difficulty with displaying and interpreting emotion.
Some useful links:
The journal ‘Nature’ on the problems with Artificial Intelligence and emotion
Charles Darwin and emotion in Scientific American:
Lisa Feldman Barrett’s book ‘How emotions are made’
A nice summary of Lisa Feldman Barrett’s book