About Face

For half a century, one theory about the way we experience and express emotion has helped shape how we practice psychology, do police work, and even fight terrorism. But what if that theory is wrong?

emotions-facial-expressions-not-related-4

The Northeastern psychologist Lisa Barrett, a modern pioneer in the study of emotion. (Photograph by Jesse Burke)

An important question continued to nag Barrett. What was the best way to determine the emotions that people are feeling? The therapist in her wanted to use the information to help her patients; her inner researcher just wanted the answer. So she dove into the emotion literature, and what she found surprised her. After reviewing all of the studies she could find, she realized that, statistically speaking, the best that scientists of emotion could do was to determine whether someone was feeling good or bad.

For Barrett, that wasn’t good enough. So she kept looking. She signed up for a physiology and cardiovascular training fellowship, to learn how to measure physiological indicators herself. And then something shocking happened. She returned to those famous cross-cultural studies that had launched Ekman’s career—and found that they were less than watertight. The problem was the options that Ekman had given his subjects when asking them to identify the emotions shown on the faces they were presented with. Those options, Barrett discovered, had limited the ways in which people allowed themselves to think.

Barrett explained the problem to me this way: “I can break that experiment really easily, just by removing the words. I can just show you a face and ask how this person feels. Or I can show you two faces, two scowling faces, and I can say, ‘Do these people feel the same thing?’ And agreement drops into the toilet.”

This exposed a fatal flaw in Ekman’s work as far as Barrett was concerned. “I mean, think about it,” she said. “When was the last time that you saw somebody win an Academy Award for going like this with fear”—at which point she mimicked for me the face in Edvard Munch’s The Scream.

Barrett wasn’t the first to question Ekman’s studies. A small handful of outlier psychologists had already begun a steady drumbeat of opposition. One had even shown he could throw the whole thing off by just showing subjects an “angry” face without including anger as a word to match it to. If people were presented with disgust or contempt instead, they happily chose one of those. But these naysayers were considered to be on the fringe. Why take them seriously when so much research already existed in Ekman’s favor?

Barrett found herself profoundly puzzled. “My experience of anger is not an illusion,” she told me. “When I’m angry, I feel angry. That’s real. But how can that be true if there’s no unique biological signature for anger?”

Barrett devoted herself to finding the answer to this question. In 1996 she accepted an assistant professorship at Boston College, where, abandoning her work as a practicing therapist, she continued to research the science of emotion. By then, brain imaging had become a useful tool, and emotion researchers were seizing on the technology to help them trace emotions back to their hotspots in the brain: fear to the amygdala, disgust to the insula, and so on. But with more reading and another training fellowship, this time in neuroscience, Barrett bumped up against the same old story. Data were mixed, conclusions uncertain. Fifty years of research in, only one thing was clear about the field: More research was necessary.

 

Barrett spent 14 years at Boston College, rising at a steady clip from assistant to associate to full professor. But by the time 2010 rolled around, her research needs and ambitions had outgrown her 1,000-square-foot lab. When Northeastern offered her a job that came along with a 3,500-square-foot multi-floor space, plus an architect to design it to her specifications, she took it. This past spring Northeastern promoted her to the status of University Distinguished Professor—the highest honor the school bestows on its faculty.

One afternoon last fall, I met Barrett at George Howell Coffee, in Newton, only a block or two from her home. While explaining exactly how the brain creates emotion—or, at least, how she believes it does—she opened a computer to show me what looked like a grainy black-and-white mishmash on the screen. “When most people look at this,” she said, “they don’t know what it is. It’s an example of experimentally induced experiential blindness. Your brain is taking in visual sensations from an object, but it can’t make sense of what it is.” The brain tries to fill in the blanks, she explained. “Some people see a lobster, some people see a bunny.”

What we were actually looking at, Barrett told me, was a bee. I couldn’t see it. But then she started clicking back and forth between that picture and a new one, which was very clearly a close-up of a bee’s body. Suddenly the grainy nonsense in the first picture snapped into bumblebee stripes. Now that I knew what I was looking at, I could see it, and for an instant everything I knew about bees flooded into my mind: their hum, their wings, their bumbling flight on a hot summer’s day, the taste of their honey. “Now,” Barrett said, “can you not see the bee? Every time you see this, you will always see the bee. Because right now your mind is adding information from your past experience to create the image of the bee.”

That, Barrett told me, is what the mind does with emotions. Just as that first picture of the bee actually wasn’t a picture of a bee for me until I taught myself that it was, my emotions aren’t actually emotions until I’ve taught myself to think of them that way. Without that, I have only a meaningless mishmash of information about what I’m feeling. In other words, as Barrett put it to me, emotion isn’t a simple reflex or a bodily state that’s hard-wired into our DNA, and it’s certainly not universally expressed. It’s a contingent act of perception that makes sense of the information coming in from the world around you, how your body is feeling in the moment, and everything you’ve ever been taught to understand as emotion. Culture to culture, person to person even, it’s never quite the same. What’s felt as sadness in one person might as easily be felt as weariness in another, or frustration in someone else.

  • Arvid Kappas

    I am very happy to read about this important issue. This is so well written, that I am sure, many readers will want to know more about this controversy – good starting points would be the work of Alan Fridlund UC Santa Barbara, who pointed out in 1986 that Darwin had never argued for a fixed link between emotions and expressions. An excellent introduction is his 1994 book “Human facial expression: An evolutionary view.”

    Jim Russell at Boston College has also worked for many years on these issues and has interesting high-profile research out there, questioning research methods and interpretations of the universality data. He also published a nice book on the topic in 1997 on the Psychology of Facial Expression with Jose-Miguel Fernandez-Dols,

    Lastly, here is a good pointer: http://www.apa.org/monitor/jan00/sc1.aspx

    • Defenestrator

      Thank you for pointing out a huge point that was mis-stated in the article — these ideas are definitely NOT new.

  • Richard Barone

    French philosopher Merleau-Ponty published Phenomenology of Perception in 1945 and “modern” psychology is just coming around to his ideas. He was right in saying that psychology has some serious limitations.

  • http://www.facebook.com/cees.timmerman Cees Timmerman

    “But in the other three piles, the Himba mixed up angry scowls, disgusted
    grimaces, and sad frowns. Without any suggestive context, of the kind
    that Ekman had originally provided, they simply didn’t recognize the
    differences that leap out so naturally to Westerners.”

    How much do the Himba watch angry, disgusted, and sad faces? Do they have TV? Did the researches ask them to demo those emotions?

    • uniquename72

      I did the experiment on myself with the latter half of the first set and the second set of pictures, and I also could not differentiate “properly” without the words being given to me. In fact, I was WAY off. And that’s after 40 years of TV watching.

      Any demonstration (or posed photo) is really just a caricature of the emotion, rather than the emotion itself. Most of our emotions are entirely internal and often changeable based on what we’re trying to convey and to whom — which is kinda the point.

  • clifflansley

    The latter part of this article suggests that our recognition/understanding of emotions has been taught. This ignores the work Ekman did in the stone age culture of Papua New Guinea with the isolated Fore Tribe. They both displayed and recognised the core emotions using the same universal triggers – loss of valued object/person = sadness; interference with goals/values = anger; etc.

    This research addresses the paragraph early in the article:

    “I can break that experiment really easily, just by removing the words. I can just show you a face and ask how this person feels. Or I can show you two faces, two scowling faces, and I can say, ‘Do these people feel the same thing?’ And agreement drops into the toilet.”

    I am interested in seeing good research that support this?

    • RebeccaSparks

      I don’t think it ignores as much as it challenges it. The problem with that Ekman (and other universalists) design close-ended tests with the goal of being able to test cross culturally, but with the close-ended test it ends up shaping the data to fit with the multiple choice answers. That result is what the test with the Himba is supposed to be “good research” that challenges it. Arvid Kappas lists some other sources, but I am not personally familiar with them.

      • clifflansley

        A key part of the work in PNG involved descriptions that correlated with the universal triggers (e.g. “Friends have come” for happiness) and the resulting expressions from genuine emotion matched the cross-cultural database …so not a multiple closed choice.
        Please can you point me to the Himba study? I am keen to learn what you are enthusiastic about and see howit counters what the scientific community has supported for so long.

        • RebeccaSparks

          If I’m reading the article we are commenting on correctly, Lisa Berett has finished the study, but is still in process trying to get it published.

          But if you’re looking for something already published, there’s plenty cited in this lit. review.

          Barrett, Lisa Feldman. “Was Darwin wrong about emotional expressions?.”Current Directions in Psychological Science 20.6 (2011): 400-406.

          These reviews do not make the bold claim that emotions are illusions. Instead, they make the more nuanced claim that emotion categories do not have firm boundaries in nature (i.e., emotions are not natural kinds). They demonstrate that behavioral, physiological, experiential and cognitive responses are highly variable within an emotion category, and this variability can be observed even in experiments explicitly designed to produce stereotypical emotional responses. Collectively, the empirical evidence points to the need to explain this observable variability in emotional responding while at the same time understand how human perceivers deal with that variability and experience or perceive discrete categories of emotion (Barrett, 2006b). Do the relatively few positive results come from methodologically superior experiments that float to the top in a sea of misguided empirical attempts? Or does highlighting those studies, while ignoring all the contrary evidence, constitute a case of confirmatory bias?”

          http://cdp.sagepub.com/content/20/6/400.full

  • Elias

    ” Barrett’s researchers would simply hand a jumbled pile of different
    expressions (happy, sad, fearful, angry, disgusted, and neutral) to
    their subjects, and would ask them to sort them into six piles.”

    They didn’t tell the subjects what they should have based the sorting on so we still don’t know WHY they placed disgust, anger and sadness in the same pile.

    • David McShane

      I wish everyone on this thread would go to Silvan Tomkins (Affect Imagery Consciousness) from whom Ekman got his ideas in the first place. To confuse affect with emotion is fatal to this research.

      • Elias

        So what is the difference between the two?

        • David McShane

          The best way to deal with this is to become familiar with Silvan Tomkins work. His conviction and demonstration that affect biases cognition predates by decades the recent confirmation of that fact by the brain imaging techniques of contemporary neuro-scienteists.

          The following may be useful. Goggle this:
          Eric Shouse · Respond to this Article. Volume 8; Issue 6; Dec. 2005. 1. AFFECT/A
          It depends upon those who knew (somewhat) Tomkins’ work but does not quote from his
          main treatment of the subject, in the four volume AFFECT IMAGERY CONSCIOUSNESS

          In capsule form one can say AFFECT IS BIOLOGY, EMOTION IS BIOGRAPHY.

          The following is quoted from the introduction to volume 1 of Tomkins’ AFFECT IMAGERY CONSCIOUSNESS (published in 1961)* written in the late 1950′s *. Given the fact that this is 50 years old gives it a quite prophetic cast. Tomkins died in 1991. He mentored both Ekman and Carroll Izard in the miulti-cultural affect recognition research.

          Following is from AFFECT IMAGERY CONSCIOUSNESS Vol 1 p.5-6.

          “It is not just consciousness in general which has been neglected, bu the role of affecdt has also been grossly underestimated. Indeed, we might speculate that the the phenomenon of consciousness might possibly never have been so neglected had the problem been restricted to what another human being thinks. It is rather knowing how he feels that has been most strikingly avoided. This is in part a consequence of the widespread taboos on affect which are learned in childhood.

          That Behaviorism slighted the role of affects is obvious, that Psychoanalysis did is less so. But if we trace the development of Freud’s theories chrdonologically, it becomes apparent that affects play a major role in his earlier papers and a successively smaller role as Psychoanalysis evolved. The affects were subordinated to the drives. As in most psychological theories, the drives were concieved to constitute the primary motivational system, and the affects played, but comparison, a lesser role in motivation. It is our contention that exactly the opposite is the case.

          In our view, the primary motivational system is the affective system, and the biological drives have motivational impact only which amplified by the affect system. Tis niew is unusual, despite the factt hat the evidence from a wide variety of sources clearly converges towards such a conclusion”.

        • Steve Shmurak

          Elias et al,

          To help clarify the difference between
          innate-biological affective signals and biographical emotion, I suggest you watch this short video (about
          12 minutes) showing the 9 innate affects in infants:

          http://www.aynrandstudies.com/jars/v8_n1/8_1toc.asp (On the second line you will find 2
          links – “PC” or “Mac”. If you click on the appropriate one you can download the
          video to your computer.)

    • Pat Field

      I agree with Elias’s concern: WHY did they place disgust, anger and sadness in the same pile?

      One
      way of thinking about it is to imagine the varied ways people respond
      to shame: some crumple and sadly accept that they are faulty beings
      (“attack self”); some put much distance between themselves and the
      shaming situation (“withdrawal”); some act like clowns (“shameless”) to
      trivialize their participation, or engage in the use of drugs or alcohol
      to forget (“denial/avoidance”); some get hostile with actual physical
      attack or with the rejection of other implied by an expression of
      disgust (“attack other”).

      When a psychologist asks, “How are
      you feeling?” some people will respond with what they think is going on:
      “I feel like she doesn’t like me.” A good therapist might continue
      the inquiry with asking the client to locate the feeling in his body.
      “I feel sick to my stomach when she looks at me that way,” he might
      respond. Continuing to probe, the psychologist might discover whether
      there are other similar events in the person’s life, where that bodily
      sensation was elicited in a relationship, and if so, explore how that
      relationship came to be and how it unfolded. And the upshot might be
      that the sick-stomach feeling functions to alert the client that
      something is amiss in the relationship, some disconnect that leaves the
      client feeling unloved, unwanted, for reasons beyond his comprehension.

      It will take much time and interaction with the psychologist
      for the client to be able to connect the feeling of “someone doesn’t
      like me, and it makes me feel sick to my stomach” with what elements
      have gone awry in some significant early relationship, and then to
      connect that experience with a current relationship that has at least
      this one aspect in common with the early one. And then to connect that
      insight with his own way of participating in relationship with
      significant others, so that he might consider developing other ways to
      relate and/or to make better choices in whom to be in relationship
      with–when he’s ready to make those changes.

      Then there’s the
      ability to tolerate the sick-to-stomach feeling as one goes about trying
      out new relationships, and using that feeling as information that one
      needs to elicit feedback from the other(s) with whom one is in
      relationship.

      What does this have to do with facial display? So
      what if I look disgusted (only slightly) when you approach me? A lot
      depends on your ability to inquire into my feelings, doesn’t it? What
      action to take? You could crumple into sadness, or run away, or take a
      drink so as not to notice, or get mad at me–or inquire.

      Inquiry–tolerating
      unpleasant feelings and adopting a healthy Interest, rather than
      dissolving into a load of negative feeling–is where learning about
      Affect Script Psychology has taken me.

      To see a brief intro to Affect Script Psychology to, please check out:
      http://tomkins.org/WhatisASP.html
      and download the two free papers at the top.

  • Matthew Quint

    This is a great piece, and a nice reminder that all research is subject to new understanding and improvement.

    There seem to me to be three things at play:

    1. Do facial responses represent an instinctual emotion? Maybe not in all cases, but no data could ever convince me that my 6 month old’s instantaneous facial expression of “yuck” when trying certain new foods is a manifested expression designed to communicate something to me rather than an instinctual internal emotional reaction to the taste of the food.

    2. Do all culture’s read facial expressions and corresponding emotions exactly the same way? I think it is pretty clear that exactly how an experiment is conducted can lead to different conclusions on this point. However, I think it is hard to argue that there isn’t some classification level — maybe only a two-to-four (may be less) facial-emotional states that wouldn’t be interpreted the same way.

    3. What’s really going on behind the facial expression-emotion connection? This is clearly the place where it seems clear to me that individual human experiences play a role. The “why” behind a facial expression and an emotion it might reflect are hugely influences by experience and context.

  • SixnaHalfFeet

    This all reminds me of Shrodingers Cat. Perhaps emotions are a quantum state that is not known until an observation is made? Or the act of observation effects the outcome?

  • Jason Barlow

    Great article – just found out his courses are coming to boston to learn these skills. http://people-intell.com/courses/course-schedule/11-emotional-skills-and-competencies-in-boston/event_details.html

  • Shane Anderson

    Isn’t the crux of Ekman’s work that emotion presents itself in universal ways, not that it is understood the same universally?… It feels like neither she nor the writer of the article have read Ekman’s work (they may have, but the article is heavily skewing stuff).
    It says it’s not a matter of semantics, but if we named a frown-y face as Orange, and smile as Banana and used fruits to label all emotions instead of Anger, Disgust etc. it feels like it would start to fall apart,
    I mean while someone may not be able to describe or interpret what they’re feeling as Happiness, a smile is ALWAYS a reaction to something positive for that person and while what is positive to one person is completely subjective for example a killer getting pleasure from someone else’s death, that their reaction to smile, I think, seems pretty standard.

    And in this day and age, for research to be dismissed seems like it says something about the research. This isn’t, after all, the 1800′s and she isn’t John Snow battling the whole word on miasma theory, it feels like there’s more than a little self-martyring going on on her behalf.