So Appy Together

Siri, the iPhone's sassy personal assistant, was just the beginning. Right now, behavioral scientists are racing to develop a new generation of apps and programs that can mimic compassion, concern, and sympathy—technology they hope we will form relationships with, even fall in love with. Janelle Nanos goes inside this brave new world to find out how our ever-smarter phones are changing what it means to be human.

By Janelle Nanos | Boston Magazine |
Illustration by Viktor Koen

Illustration by Viktor Koen

Monday morning begins with the chime of bells. Blinking awake, I turn toward the noise, pawing at my bedside table in search of my phone. With a quick tap the bells are silenced, as if someone has abruptly cut the ropes in the belfry. I remove the sleep sensor from my forehead and adjust my glasses, scanning through the data from last night to check my REM. “What’s on the agenda for today?” I ask. “Four meetings ahead of you, two assignments due soon,” my phone says, then pings me with my Daily Challenge text message. Today I’m told to try to take 5,000 steps. Totally doable, I think, then reach for my body-monitoring armband and slide it up onto my biceps. My phone connects to it via Bluetooth, and begins registering my movement as I head to the kitchen. I open the fridge, grab the milk, then pour a bowl of cereal, pulling up my phone’s diet tracker to scan the bar codes on each container for an accurate calorie count.

“We’d better finish the milk — it spoils tomorrow,” my husband says, and I realize it’s the first time we’ve spoken this morning. I grimace apologetically, and flip my phone face-down on the table.


I love my smartphone. It’s become a second brain in my pocket that’s changed how I process information. It’s with me every waking moment — and the sleeping moments, too — tracking my daily habits. And through my constant e-mail and Facebook activity, and the personal documentation of my life via Twitter and Instagram photos, it’s become the lens through which I see the world. All day long, I find myself instinctively reaching for my phone, using it as a tool to validate my existence.

But lately, my smartphone and I have taken our relationship to the next level. I provide it with ever-more-intimate details about my life. Last year, for example, I set a few goals for myself. I wanted to lose some weight, save money, and run a half marathon. With only a few app downloads, my phone became a trainer, life coach, and confidant. It now knows what I eat, how I sleep, how much I spend, how much I weigh, and how many calories I burn (or don’t) at the gym each day. It’s gotten to the point where my phone now somehow knows more about me than anyone else in the world, including my own darling husband. My gadget has become a tiny black mirror, reflecting back how I see myself. Which means things are getting more complicated between us.

Lately, I’ve found myself trying to outsmart my smartphone, fudging my calorie intake when I’ve gone overboard on dessert, or hiding a credit card from my personal-finance app so it doesn’t know about my occasional spending sprees. And then I catch myself: It’s a damn phone. This is insane!

What’s going on here? How is it that I’ve come to feel accountable to a device — my device — one that works for me, not the other way around? I know I can’t be the only one asking these questions, as 30 percent of Americans now own smartphones (and one-third of them have downloaded apps to help them monitor their lives). Right now, something called “behavioral scientists” are hard at work dreaming up technologies to make our phones more and more human in the way they interact with us — and to encourage us to build relationships with them. It turns out that Siri, the iPhone’s wildly popular new personal assistant, is just the beginning. Where are we headed? I wanted to know before things got any weirder between my phone and me.


Like I said, I lie to my phone. On the days when I’m exceeding my calorie goal in my weight-loss app, I’ll sometimes tweak the numbers. Or I’ll just avoid using the app at all that day.

But I don’t enjoy the feeling that I’m cheating on my phone. So I reach out to Sherry Turkle, an MIT professor and author of the recent book Alone Together: Why We Expect More from Technology and Less from Each Other.

Appy Together

Illustration by Viktor Koen

Turkle, a sociologist and clinical psychologist, has spent the past two decades exploring the relationship between humans and robots. She’s currently researching Siri and the way its users have, in only a few months’ time, come to think of it as a nonjudgmental best friend. Siri is a “cultural preparation for a kind of intimacy with our machines that will take us to a new level,” Turkle explains. As humans, she says, we’re programmed to anthropomorphize objects. But until now, these objects haven’t been programmed to love us back.

“We’re at a moment of temptation,” she says. “We’re entering into a whole new level of relationship with inanimate objects. And they’re not just inanimate objects that we can project on. We have objects that have little minds of their own.”

So that’s why I lie to my phone about my calories, because I feel like it thinks less of me when I screw up?

“The calorie counter externalizes our self-­expectations,” Turkle says, explaining that these devices are becoming extensions of us. “We don’t want to have some piece of ourselves punishing us.”

I thought these apps were supposed to be giving us control over our lives. But here we are, feeling beholden to them, feeling guilty in their presence. “Nagging is still nagging, whether it comes from your phone or your mom,” says Jessie Gruman, a social psychologist who heads the Center for Advancing Health, a patient-advocacy group out of Washington, DC. Gruman is a four-time cancer survivor who’s tried nearly every program on the market to help keep her weight up after she lost a portion of her stomach to the disease. But she gets so frustrated with the apps — with how time-consuming they are, or how generally annoying they become — that she’s deleted more than she can count. Because we think of our phones as tools that serve us, it’s disconcerting to find ourselves responding to their demands, she says. “We like our relationships with our devices to remain constant and uncomplicated.”


One unseasonably warm evening in January, I arrive at NuVu studio in Central Square to meet a group of what are known as self-trackers — people who use data and technology to help make sense of their lives. About two dozen men and a few women, mostly in their thirties, sit in red and yellow swivel chairs. Lounging on sofas in the back is a group of teenagers who attend NuVu as part of a magnet high school program.

Self-trackers measure, time, and count just about everything they do, keeping track of it all as a way to improve their health and their existence. One presenter at the meeting explains how he combined self-tracking with holistic principles to speed his recovery from a backbreaking fall. The teenagers describe how they’re trying to increase productivity by snoozing in two-hour chunks throughout the day, a technique called polyphasic sleep. They looked tired.

Modern technology has made it a lot easier to keep track of our lives, which is probably why more and more of us seem to be doing it. Who goes through the trouble of balancing a checkbook anymore? The concept is absurd. Instead, we log onto our bank’s website and get a snapshot of our activity. Or we hand over our banking passwords to sites like Mint for insight into our overall financial situation. In a way, we’ve all become self-trackers. But as you might expect, some of us are more hard-core than others.

So Appy Together

Illustration by Viktor Koen

A few years ago, some very smart people started to wonder what would happen if we tracked ourselves as closely as do the marketers who scarf up the digital breadcrumbs we leave with our every move on the Internet. (Our credit card purchases, GPS usage, and DVR activity are just a few of the other ways corporate America keeps tabs on us.) Could the same kind of data big business is always collecting help us learn more about ourselves? That question gave rise to the Quantified Self (QS) movement, a term coined by Wired editors Gary Wolf and Kevin Kelly in 2008. QS was modeled on the Homebrew Computer Club, a group of Silicon Valley tech hobbyists whose biweekly gatherings led to the creation of the personal computer (Steve Jobs was a member). Early QS adherents made elaborate Excel spreadsheets or developed their own computer codes to keep track of life in minute detail. They measured things like precisely how much caffeine they needed to hit maximum productivity, or how diet and sleep affected their athletic performance. One member discovered that eating half a stick of butter each day enhanced by 5 percent his ability to do simple math problems. Life is messy, they reasoned, whereas data is efficient, rational, and beautiful.

The group at NuVu is one of more than 40 QS chapters that meet regularly in 18 countries. And, just as the Homebrew Computer Club brought forth the PC and Mac, QS has influenced the creation of apps and gadgets that have brought tracking to the masses. A movement that was once the domain of hackers and techies is now available to all of us, and it’s spreading. MobiHealthNews estimates that there will be 13,000 health-related apps in the iTunes store by this summer. And according to Wellesley-based BCC Research, the health technology industry, which was valued at more than $7.4 billion in 2011, is projected to more than double to nearly $17.5 billion by 2016. Body sensors littered the floor of the International Consumer Electronics Show in Las Vegas in early January. That same month, a little company called Nike unveiled a new activity-tracking device — the FuelBand.

“Starting QS was about anticipating that this boom was coming, and wanting to produce a context for intelligence and reflection,” Gary Wolf tells me when I get him on the phone in California. He says QS is not about finding answers, per se, but about asking questions and learning more about yourself in the process. He goes on to explain that this convergence of technology and personal data is creating a “new way of being human, and it’s something that we should think and talk about and work out together.”


A new way of being human? That’s certainly not what I thought I was signing up for when I began my own self-tracking about a year ago. I downloaded one of the most popular health apps on iTunes, RunKeeper, which was designed by the local startup FitnessKeeper. As I jog, it tracks my pace and uses my phone’s GPS coordinates to calculate my distance, while its internal coach gives me my splits and tallies my calories burned. RunKeeper’s been called the gateway drug for self-tracking, and after getting the initial bit of data, I did feel a dopamine surge in my brain. As I kept using the app, I got hooked. I’d receive e-mails when I hit a new distance goal, and would share my runs on Facebook, where friends would cheer me on, adding to the warm-fuzzy feeling. Later, I learned that I was experiencing what researchers call the Hawthorne effect: the idea that test subjects improve upon an observed habit simply because it’s being watched. Socrates famously said that an unexamined life isn’t worth living. And sure enough, I began to feel like I needed to run with my phone for my workouts to really count.

Then last summer, I noticed that all of the maps of my runs were being logged online, and made searchable by Google. Realizing that someone could easily deduce where I lived, I quickly changed my settings to private. It was a bit of a buzzkill, and feeling slightly betrayed, I began jogging without my phone. But I got off easy. This past July, users of the Fitbit, a clip-on device that monitors your physical movement, were surprised to learn that after logging their sexual activity (the average person burns 150 to 200 calories per go, so why not count it?), the information was available online, including how long, how vigorous, and how often they got busy.

So privacy is definitely an issue as personal technology becomes an even bigger part of our lives. But I had another question: What happens when the QS movement — which began as a kind of protest against the commercial chokehold on data — becomes a commercial enterprise in itself? Best Buy is now stocking these kinds of sensors, for Christ’s sake. Is the introspection component of QS, so important to Wolf, getting lost?

After the self-tracking meeting in Central Square, I share some ice cream at Toscanini’s with Michael Nagle, an MIT grad who heads the QS chapter here, and Joshua Kauffman, a researcher at Harvard’s Graduate School of Design, who was an early member of the QS movement. I ask them about some of the critiques of QS — that it’s self-absorbed or narcissistic, and that the time spent tracking could be better spent actually living life. Kauffman shakes his head. “The most intelligent rebuttal to the Quantified Self that you hear most often is that it’s just like GPS,” Kauffman said. “If you’re too reliant on GPS for direction, you lose your sense of space. It’s the same thing with the Quantified Self, that if you’re looking too much at external sensing and indicators of who you are, you’ll lose your capacity for self-reflection.

“But the answer to that is the Quantified Self enhances your capacity for self-reflection,” he continues. “And if anything, it enhances your intuition. If you can use a tool that can help you understand the relationship between your consumption of ice cream and your body mass index, eventually you’ll be able to look at the ice cream and go, ‘I know what that means on my graph. It’s not just an abstract object, it has a relationship. It affects my biological system.’”

But doesn’t that take all the fun out of ice cream? I ask, looking down at my half-eaten scoop of toasted coconut–macadamia nut chip.

“It reinserts the fun,” he says. “There’s not the feeling of guilt, there’s the feeling of honesty.”


When Henry Ford created the Model T, it didn’t have a dash-board. Early drivers had no idea how fast they were going or how much gas was in the tank. We’ve been functioning in much of the same way when it comes to monitoring our health. Aside from the occasional visit to the doctor, we’ve had little way of really knowing what was going on underneath our hood. But now technologists see opportunity in a brave new world where the screens on our phones are tiny dashboards, crunching our data and tipping us off when our bodies are running out of gas, in need of a tune-up, or somehow malfunctioning.

And Frank Moss, the former director of MIT’s Media Lab and current head of the New Media Medicine Group, says Boston is poised to lead the way in developing those dashboards. When I visit him in his office at Cambridge’s Bluefin Labs, Moss tells me that our phones are on the way to becoming “personal digital nervous systems” that monitor our activities. Our city may have lost out in the race to develop the personal computer, but he says the consumer health revolution is our chance for redemption: “This is the biggest challenge and opportunity of the 21st century.”

He’s right. You can’t throw a rock in this town without hitting a self-tracking startup. Need a way to monitor your weight? Download the Lose It! app, created in Copley Square. Feeling groggy in the mornings? Pick up the new mobile headband from Zeo, the Newton-based maker of sensors that help you analyze your sleeping patterns. Focused on wellness? The Daily Challenge from South End–based MeYou Health sends you e-mails and texts that help you set tiny goals each day. Looking for motivation to get to the gym? Dozens of developers in Boston are building apps that will get creative in finding ways to nudge you toward the treadmill: Are you motivated by challenges, games, or rewards? Do you respond to positive or negative feedback?

Jason Jacobs is one of the developers in the race to build the ultimate dashboard. Jacobs is the founder of FitnessKeeper, maker of the RunKeeper app that first got me hooked on self-tracking, so I meet with him to find out where things are headed next. He says the future is in creating a single program that puts in one place all of the personal data that right now is being kept in different devices and apps. Without that, he says, it’s up to the user to make sure all of her various gadgets and programs are accurate, up-to-date, and (er) honest.

So this past June, RunKeeper launched a tool called the Health Graph, which lets data junkies sync up all of their numbers in one place. Just five months later, the company received $10 million, much of it in venture funding from Spark Capital, the local outfit that has supported Twitter, Tumblr, and Foursquare. This is exciting stuff for Jacobs, and as he talks about it, he touches his hands to his temples, as if he’s trying to keep his mind from being blown. “If you’ve got all this data in one place, you can build a self-optimizing system,” he says. “And then if it knows all my data and what my goals are, it’s like, Tell me what to do. The system can become more predictive and prescriptive over time.” I like running with my phone, but now it’s going to tell me what to do? Technology is supposed to allow us to do more in our lives without thinking about it, not do the thinking for us, right?

This gets even more complicated when it’s not just our physical health that these devices are monitoring, but our emotional health, as well., a team spawned from MIT’s Media Lab, has developed an app that notices when you’re not texting as much with friends or haven’t left home for days at a time, and can tip you or your doctors off to the onset of depression. And the Waltham-based tech firm Affectiva, another Media Lab spinoff, has a wristband called the Q Sensor that measures your emotional peaks and valleys throughout the day. “It tells you things that your body is feeling that you’re ignoring,” says Affectiva cofounder Rosalind Picard, who also heads MIT’s Affective Computing group. Right now, the Q Sensor is being used to help children with autism better monitor their emotions, but Picard says it could someday become a consumer device.

So yes, in theory, my phone could one day pick up on the fact that my stress levels are rocketing at work, and suggest that I go out for a walk to take a break and catch up on the exercise I’ve been slacking on…so long as it’s not a bar I’m walking to (which it would know about thanks to GPS), because, really, that isn’t the best way to deal with my problems, is it? And I’m just supposed to smile and obey?

“It’s dizzying for the average consumer to sort this out,” says Joe Kvedar, a physician who runs the Center for Connected Health at Partners HealthCare. Kvedar’s a bona-fide self-tracker — he clips a pedometer to his slippers when he wakes up in the morning and just installed a treadmill desk in his home in Medfield. But he’s frustrated by the lack of insightful layers on top of the data to help us make sense of it all.

What apps need, Kvedar says, is something called a feedback loop to both explain things to the user and provide a degree of accountability. Why? Let’s return to the car dashboard for a moment. A speedometer should be enough to stop us from speeding, but researchers have found that one of the best ways to get people to actually slow down is to post those signs on the side of the road that tell you how fast you’re going. Kvedar thinks a tool like Siri is heading in the right feedback-loop direction. “I’m just waiting for someone to do an integration between Siri and some kind of an activity monitor,” Kvedar tells me. “I think that could be very powerful.”

Kvedar’s not alone. Nearly every developer I spoke with said that Siri may be a game changer when it comes to mobile health. And behavioral scientists and designers of these products are already busy building algorithms to implant into Siri’s brain. In so doing, they’re probing our psyches to find ways to get us to trust and build relationships with these tools.

Behavioral scientist Betsy Barbeau is one of these digital “neurosurgeons.” As the chief science officer at Healthrageous, the commercial arm that grew out of the Center for Connected Health, she’s designing artificial intelligence technology that will better understand us humans. People who sign up for Healthrageous are guided by a virtual coach that’s been programmed using a technique called motivational interviewing. The goal is to create an empathetic, almost therapeutic relationship between the patient and the technology. “There’s no wagging of the finger, there’s no, ‘You blew it this week,’” Barbeau explains. “It’s the idea that someone is watching you, checking in, and giving you an ‘atta girl’ if you met your goals.” That “someone” is actually a something, but who’s counting? I ask Barbeau what would happen if I lied to the coach about my calorie intake. She laughs. “Well, I could see it saying something like, ‘I see your activity level has been very high but your weight level hasn’t been changing.’ It would be an opener for further reflection.”

Knowingly cultivating an empathetic relationship with your smartphone is downright creepy. But the thing is, virtual coaches actually work. A study in the Journal of Medical Internet Research, published in January and coauthored by Kvedar and Timothy Bickmore, a computer science professor at Northeastern University, found that having a virtual coach can steer people toward better health. In the study, they provided 70 adult Bostonians with pedometers and access to a website where they could track their steps. Half of them were also able to work with an animated coach on the site that offered personalized feedback on how they were doing. At the end of the experiment, those who were coached walked an average of a half mile more per day than those who weren’t.

Bickmore says these programs are trying to replicate the communication between doctors and patients. The coach can have a caring face, he explains. “We shift its pitch and slow its voice down so it sounds more concerned. We’re trying to acknowledge the emotional dimension of what’s going on.” He hasn’t yet researched the degree to which people feel bonded to these coaches, but he’s hopeful. “A lot of time at the end of the studies,” he says, “we have participants who don’t want the study to end, and don’t want us to take the character away.”


Moss, the head of New Media Medicine Group, says it’s best to think of our phones not as coaches but as “partners” working alongside us. He says it’s smart to begin thinking about the emotional interactions we have with our phones now, before we get to the point where we’re breaking up with devices or wondering why they aren’t returning our calls. But he also cautions that the fear that we’ll become cyborgs shouldn’t deter us from charging full bore into the digital health space. For one thing, he argues, the more data we collect about ourselves, the more it can be used to help feed into the population, giving us a far more accurate way to shape public health policy.

I ask Sherry Turkle, the MIT professor, if she can foresee a day when she views her phone as a partner.

She pauses. “I get migraines,” she says — the close-the-drapes kind that keep her in bed for days. If a phone had the capacity to measure her temperature, pupil dilation, and galvanic skin response — three things that indicate the onset of a migraine — and then tell her to take a pill to fend off the massive headache, she would absolutely want it. It would be akin to having a hero rescue a damsel from the path of an oncoming train.

When I ask Turkle how she’d feel about a device that could do all that for her, she lets out a laugh.“Could youimagine the feelings of intense connection and gratitude, bordering on profound love?” she says. “It would be almost impossible not to love them.”

At which point, I get the chills. You see, it makes me anxious to think about it, but the truth is, I know I want that, too. The allure of knowing so much about myself — even if it means being inextricably linked to a tiny gadget in my pocket — is too strong. When the time comes, I’m going to surrender.

Turkle says that as the technology advances, we’re going to have to figure out how to deal with such a reality. “We have to learn how to exert a certain kind of discipline to remember that these devices don’t care about us. That they don’t love us. That they’re not looking out for us. That there’s nobody home.”

Welcome to the next stage of human evolution: playing out in the palm of our hands.

Source URL: