About Face

For half a century, one theory about the way we experience and express emotion has helped shape how we practice psychology, do police work, and even fight terrorism. But what if that theory is wrong?

By | Boston Magazine |
emotions-facial-expressions-not-related-1

Photographs by Jesse Burke

Forty-six years ago a young San Francisco–based cowboy of a psychologist named Paul Ekman emerged from the jungle with proof of a powerful idea. During the previous couple of years, he had set out trying to prove a theory popularized in the 19th century by Charles Darwin: that people of all ages and races, from all over the world, manifest emotions the same way. Ekman had traveled the globe with photographs that showed faces experiencing six basic emotions—happiness, sadness, fear, disgust, anger, and surprise. Everywhere he went, from Japan to Brazil to the remotest village of Papua New Guinea, he asked subjects to look at those faces and then to identify the emotions they saw on them. To do so, they had to pick from a set list of options presented to them by Ekman. The results were impressive. Everybody, it turned out, even preliterate Fore tribesmen in New Guinea who’d never seen a foreigner before in their lives, matched the same emotions to the same faces. Darwin, it seemed, had been right.

Ekman’s findings energized the previously marginal field of emotion science. Suddenly, researchers had an objective way to measure and compare human emotions—by reading the universal language of feeling written on the face. In the years that followed, Ekman would develop this idea, arguing that each emotion is like a reflex, with its own circuit in the brain and its own unique pattern of effects on the face and the body. He and his peers came to refer to it as the Basic Emotion model—and it had significant practical applications. In the late 1960s, for example, Ekman realized that he could detect the microexpressions of emotion that appear on the face of a liar. Anybody trained in how to properly recognize these microexpressions, he would later argue, could detect a liar 70 percent of the time. He published his first article on the subject in 1969, and three months later the CIA came knocking, eager to learn more.

So began a meteoric rise to fame. Since that first article, Ekman has consulted for not only the CIA but also the FBI, the Department of Homeland Security, the New York Police Department, and the Transportation Security Administration, which has spent more than a billion dollars training its airport agents in techniques based on Ekman’s theories. He’s published scores of influential papers and books, and his findings have been verified and expanded upon in hundreds of studies. In 2001 the American Psychological Association named him one of the most influential psychologists of the entire 20th century. And in 2009 Time named him one of the 100 most influential people in the world.

Ekman is a giant in his field, in other words. His ideas have powerfully shaped the science of emotion for half a century. But here’s the thing: What if he’s wrong?

 

“Honestly, this is going to sound terrible,” Lisa Barrett told me when I asked her about Ekman and his original study. “But at first, when I read that work, I thought, Well, nobody can take this seriously. This can’t possibly be right. It’s too cartoonish.” 

Barrett is a professor of psychology at Northeastern, and for years she’s been troubled by Ekman’s ideas. People don’t display and recognize emotions in universal ways, she believes, and emotions themselves don’t have their own places in the brain or their own patterns in the body. Instead, her research has led her to conclude that each of us constructs them in our own individual ways, from a diversity of sources: our internal sensations, our reactions to the environments we live in, our ever-evolving bodies of experience and learning, our cultures.

This may seem like nothing more than a semantic distinction. But it’s not. It’s a paradigm shift that has put Barrett on the front lines of one of the fiercest debates in the study of emotion today, because if Barrett is correct, we’ll need to rethink how we interpret mental illness, how we understand the mind and self, and even what psychology as a whole should become in the 21st century.

Twenty-one years ago, Barrett had no idea she’d be wading into this debate. In 1992 she was just another graduate student studying clinical psychology at the University of Waterloo—the MIT of Canada. She had every intention of becoming a therapist. True, she was unusually engrossed in the research side of her program. But the general type of study she was doing, exploring how people’s perceptions of themselves can lead to either anxiety or depression, was the perfect choice for a future shrink.

Times were hard for her back then. Her marriage was in tatters, her thesis adviser had just left town, she was in the middle of grueling comprehensive exams, and every time she attempted to run studies necessary for her research, they failed. In one particularly troublesome experiment, no one she tested could seem to distinguish anxiety from depression—even though differentiating the two was the entire point of the experiment. “If they reported feeling sad,” Barrett told me, “they also felt anxious. And if they reported feeling anxious, they also felt sad. And I thought, Well, can’t they tell the difference?” Every paper she read told her that they were two different emotional states of mind—one based in fear, the other in sadness.

It was a puzzle. Colleagues suggested that it was probably just normal statistical error and urged her to move on. But she couldn’t drop it. She’d already triple-checked her study design and crosschecked her subjects. What was left? She eventually decided it had to be the testing measures that she and others had been using. These, she realized, were actually useless when it came to assessing whether a person felt bad and worked up about it (anxious) or bad and lethargic about it (depressed). And that, in turn, called into question many of the supposedly successful studies that her work had been designed to replicate. Barrett wrote her work up, defended her Ph.D., did a clinical internship at the University of Manitoba, and then packed up for University Park, Pennsylvania, to start life as an assistant professor of psychology at Penn State.

emotions-facial-expressions-not-related-4

The Northeastern psychologist Lisa Barrett, a modern pioneer in the study of emotion. (Photograph by Jesse Burke)

An important question continued to nag Barrett. What was the best way to determine the emotions that people are feeling? The therapist in her wanted to use the information to help her patients; her inner researcher just wanted the answer. So she dove into the emotion literature, and what she found surprised her. After reviewing all of the studies she could find, she realized that, statistically speaking, the best that scientists of emotion could do was to determine whether someone was feeling good or bad.

For Barrett, that wasn’t good enough. So she kept looking. She signed up for a physiology and cardiovascular training fellowship, to learn how to measure physiological indicators herself. And then something shocking happened. She returned to those famous cross-cultural studies that had launched Ekman’s career—and found that they were less than watertight. The problem was the options that Ekman had given his subjects when asking them to identify the emotions shown on the faces they were presented with. Those options, Barrett discovered, had limited the ways in which people allowed themselves to think.

Barrett explained the problem to me this way: “I can break that experiment really easily, just by removing the words. I can just show you a face and ask how this person feels. Or I can show you two faces, two scowling faces, and I can say, ‘Do these people feel the same thing?’ And agreement drops into the toilet.”

This exposed a fatal flaw in Ekman’s work as far as Barrett was concerned. “I mean, think about it,” she said. “When was the last time that you saw somebody win an Academy Award for going like this with fear”—at which point she mimicked for me the face in Edvard Munch’s The Scream.

Barrett wasn’t the first to question Ekman’s studies. A small handful of outlier psychologists had already begun a steady drumbeat of opposition. One had even shown he could throw the whole thing off by just showing subjects an “angry” face without including anger as a word to match it to. If people were presented with disgust or contempt instead, they happily chose one of those. But these naysayers were considered to be on the fringe. Why take them seriously when so much research already existed in Ekman’s favor?

Barrett found herself profoundly puzzled. “My experience of anger is not an illusion,” she told me. “When I’m angry, I feel angry. That’s real. But how can that be true if there’s no unique biological signature for anger?”

Barrett devoted herself to finding the answer to this question. In 1996 she accepted an assistant professorship at Boston College, where, abandoning her work as a practicing therapist, she continued to research the science of emotion. By then, brain imaging had become a useful tool, and emotion researchers were seizing on the technology to help them trace emotions back to their hotspots in the brain: fear to the amygdala, disgust to the insula, and so on. But with more reading and another training fellowship, this time in neuroscience, Barrett bumped up against the same old story. Data were mixed, conclusions uncertain. Fifty years of research in, only one thing was clear about the field: More research was necessary.

 

Barrett spent 14 years at Boston College, rising at a steady clip from assistant to associate to full professor. But by the time 2010 rolled around, her research needs and ambitions had outgrown her 1,000-square-foot lab. When Northeastern offered her a job that came along with a 3,500-square-foot multi-floor space, plus an architect to design it to her specifications, she took it. This past spring Northeastern promoted her to the status of University Distinguished Professor—the highest honor the school bestows on its faculty.

One afternoon last fall, I met Barrett at George Howell Coffee, in Newton, only a block or two from her home. While explaining exactly how the brain creates emotion—or, at least, how she believes it does—she opened a computer to show me what looked like a grainy black-and-white mishmash on the screen. “When most people look at this,” she said, “they don’t know what it is. It’s an example of experimentally induced experiential blindness. Your brain is taking in visual sensations from an object, but it can’t make sense of what it is.” The brain tries to fill in the blanks, she explained. “Some people see a lobster, some people see a bunny.”

What we were actually looking at, Barrett told me, was a bee. I couldn’t see it. But then she started clicking back and forth between that picture and a new one, which was very clearly a close-up of a bee’s body. Suddenly the grainy nonsense in the first picture snapped into bumblebee stripes. Now that I knew what I was looking at, I could see it, and for an instant everything I knew about bees flooded into my mind: their hum, their wings, their bumbling flight on a hot summer’s day, the taste of their honey. “Now,” Barrett said, “can you not see the bee? Every time you see this, you will always see the bee. Because right now your mind is adding information from your past experience to create the image of the bee.”

That, Barrett told me, is what the mind does with emotions. Just as that first picture of the bee actually wasn’t a picture of a bee for me until I taught myself that it was, my emotions aren’t actually emotions until I’ve taught myself to think of them that way. Without that, I have only a meaningless mishmash of information about what I’m feeling. In other words, as Barrett put it to me, emotion isn’t a simple reflex or a bodily state that’s hard-wired into our DNA, and it’s certainly not universally expressed. It’s a contingent act of perception that makes sense of the information coming in from the world around you, how your body is feeling in the moment, and everything you’ve ever been taught to understand as emotion. Culture to culture, person to person even, it’s never quite the same. What’s felt as sadness in one person might as easily be felt as weariness in another, or frustration in someone else.

So there’s no such thing as a basic emotion? It sounds crazy. But this is where all sorts of brain science is headed. Researchers once assumed that the brain stored specific memories, but now they’ve realized that there is no such stash to be found. Memories, the new science suggests, are actually reconstructed anew every time we access them, and appear to us a little differently each time, depending on what’s happened since. Vision works in a similar way. The brain, it turns out, doesn’t consciously process every single piece of information that comes its way. Think of how impossibly distracting the regular act of blinking would be if it did. Instead, it pays attention to what you need to pay attention to, then raids your memory stores to fill in the blanks.

In the spring of 2006, Barrett published a pair of controversial papers. The first, which appeared in the inaugural issue of Perspectives on Psychological Science, compiled every argument and question she’d found during the previous 16 years that challenged the “natural-kind view”—her term for Ekman’s theory. “The natural-kind view has outlived its scientific value,” she declared, “and now presents a major obstacle to understanding what emotions are and how they work.” The second paper, which appeared in Personality and Social Psychology Review, sketched out a richer research agenda and promised to resolve the inconsistencies and address the problems that had emerged in the prior 50 years of emotion research.

The papers provoked strong reactions. In many quarters, Barrett was angrily attacked for her ideas, and she’s been the subject of criticism ever since. “I think Lisa does a disservice to the actual empirical progress that we’re making,” says Dacher Keltner, a Berkeley psychologist who studies positive emotions and has debated publicly with Barrett in the past. “There are a zillion data points on a perspective that conforms to Ekman, and the alternative has yet to be documented convincingly.” Keltner told me that he himself has coded thousands of facial expressions using Ekman’s system, and the results are strikingly consistent: Certain face-emotion combinations recur regularly, and others never occur. “That tells me, ‘Wow, this approach to distinct emotions has real power,’” he says.


emotions-facial-expressions-not-related-2

The photographs above were used by Paul Ekman in the 1960s and 1970s as he conducted his pioneering cross-cultural studies of emotion. Subjects were shown the photographs and asked to match what they saw in them to a list of emotions, or stories about emotions. Ekman reported striking results: People all over the world matched the same faces to the same emotions, suggesting that we all express basic emotions in the same way, regardless of age, gender, or culture. From left, the emotions shown above, as described by Ekman, are anger, contempt (modeled by Ekman himself), disgust, surprise, sadness, happiness, and fear.


But Barrett’s papers and her subsequent work have also attracted praise. According to the Stanford psychologist James Gross, they have made her “one of the most important contemporary figures in the field.” Michael Spivey, a professor of cognitive science at the University of California, Merced, contends that Barrett is “almost single-handedly taking the field of emotion research into the 21st century.” In 2007, too, the National Institutes of Health unanimously awarded Barrett a Pioneer Grant, as part of an initiative aimed at funding high-risk-high-payoff ideas in science. Barrett received $3.9 million, to be used in her emotion studies however she saw fit. She is only the second psychologist ever to have won the prize.

For his part, Ekman likes to remain above the fray these days. “If you can show Ekman’s wrong,” he said when I asked him about Barrett and others who have attacked his ideas, “you’ll become famous. I’m not saying that’s their motive. I’m only describing the reality.”

 

Ekman reached the peak of his fame in the years following 2001. That’s the year the American Psychological Association named him one of the most influential psychologists of the 20th century. The next year, Malcolm Gladwell wrote an article about him in the New Yorker, and in 2003 he began working pro bono for the TSA. A year later, riding the updraft of success, he left his university post and started the Paul Ekman Group, a consulting firm that he still runs today, which has taught police departments, law-enforcement agencies, and intelligence-gathering services how to read faces for emotions. David Matsumoto, an Ekman protégé, has provided similar services to the U.S. Bankruptcy Court Federal Judges, the U.S. Court of Appeals, and even to doctors at the Mayo Clinic through his own California-based company, Humintell, which he founded in 2009. Both Ekman and Matsumoto have created websites on which you can buy introductory face-reading kits for less than $30.

Ekman and his ideas, in other words, still have a powerful influence on society at large, and scientists in a variety of disciplines continue to rely on his research. But doubts are emerging. In 2010, for example, forensic psychologists at the University of British Columbia reported in Legal and Criminological Psychology that when they’d actually gone looking for the link between the microexpressions of liars and certain universal emotions (the kind of link upon which Ekman’s theory depends), they hadn’t been able to find it. The authors wrote, “We, like most everyone else, it seemed, presumed the firm empirical foundation of the validity of microexpressions in relation to deception. But in 2006, despite reading of anecdotal evidence, we were unable to find any published empirical research on the phenomenon.”

That same year, even the U.S. Government Accountability Office weighed in, releasing a report suggesting that SPOT, the TSA’s behavior-detection program, might have been launched without proper scientific confirmation that its underlying premise was valid. A subcommittee hearing and several follow-up reports later, the program’s status remains uncertain, despite $200 million that has been spent annually on it since 2003. “If governments are buying into notions that are not scientifically sound or empirically supported,” says Maria Hartwig, a deception researcher at the John Jay College of Criminal Justice who testified at the hearing, “then I think the money’s being wasted.”

emotions-facial-expressions-not-related-3

Photographs by Jesse Burke

Barrett recently decided to take on Ekman’s ideas directly, by sending a small research team to visit the isolated Himba tribe in Namibia, in southern Africa. The plan was this: The team, led by Maria Gendron, would do a study similar to Ekman’s original cross-cultural one, but without providing any of the special words or context-heavy stories that Ekman had used to guide his subjects’ answers. Barrett’s researchers would simply hand a jumbled pile of different expressions (happy, sad, fearful, angry, disgusted, and neutral) to their subjects, and would ask them to sort them into six piles. If emotional expressions are indeed universal, they reasoned, then the Himba would put all low-browed, tight-lipped expressions into an anger pile, all wrinkled-nose faces into a disgust pile, and so on.

It didn’t happen that way. The Himba sorted some of the faces in ways that aligned with Ekman’s theory: smiling faces went into one pile, wide-eyed fearful faces went into another, and affectless faces went mostly into a third. But in the other three piles, the Himba mixed up angry scowls, disgusted grimaces, and sad frowns. Without any suggestive context, of the kind that Ekman had originally provided, they simply didn’t recognize the differences that leap out so naturally to Westerners.

Barrett, Gendron, and two others wrote a paper based on this study, which Barrett considers one of her most important to date, and submitted it to Science this past December, with high hopes for its publication. “What we’re trying to do,” she told me, “is to just get people to pay attention to the fact that there’s a mountain of evidence that does not support the idea that facial expressions are universally recognized as emotional expressions.” That’s the crucial point, of course, because if we acknowledge that, then the entire edifice that Paul Ekman and others have been constructing for the past half-century comes tumbling down. And all sorts of things that we take for granted today—how we understand ourselves and our relationships with others, how we practice psychology and psychiatry, how we do police work and gather intelligence—will have to change.

 

This past January, I visited Barrett at her home. She was tired. Two colleagues had just died in quick succession, and she’d gone to both of their memorials the week before. She was scheduled to fly out again in a couple of days for a debate at a psychology conference in New Orleans. And she’d just heard back from Science about the Namibia paper. The news wasn’t good: They’d rejected it.

“I felt fed up,” she told me, describing her reaction. “I just felt like, Why am I banging my head against a wall? Life is short. What the hell am I doing? Clearly people don’t give a shit about data, because if they did, I wouldn’t have this battle on my hands.” She paused. “I did feel that way for about 10 minutes. And then I took a step back and said, ‘Okay, I’ve seen reviews like this before.’”

Barrett showed me the feedback she’d received on the manuscript from two anonymous peer reviewers. One had been positive, but the other had written a scathing two-page response that had started by declaring the work “unjustified” and gone downhill from there. Barrett told me she suspected the second reviewer had misunderstood her statistical methods and had based several of his or her arguments on misrepresented sources. Nevertheless, she’s now adopted a philosophical stance toward it all. “Science is about persevering in the face of ambiguity and, oftentimes, adversity,” she says. “And the data, in the end, will point the way.”

It’s early days yet. Barrett’s theory is still only in its infancy. But other researchers are beginning to take up her ideas, sometimes in part, sometimes in full, and where the science will take us as it expands is impossible to predict. It’s even possible that Barrett will turn out to be wrong, as she herself acknowledges. “Every scientist has to face that,” she says. Still, if she is right, then perhaps the most important change we’ll need to make is in our own heads. If our emotions are not universal physiological responses but concepts we’ve constructed from various biological signals and stashed memories, then perhaps we can exercise more control over our emotional lives than we’ve assumed.

“Every experience you have now is seeding your experience for the future,” Barrett told me. “Knowing that, would you choose to do what you’re doing now?” She paused a beat and looked me in the eye. “Well? Would you? You are the architect of your own experience.”

Source URL: http://www.bostonmagazine.com/news/article/2013/06/25/emotions-facial-expressions-not-related/