Genius Composition
Illustration by Harry Campbell
The MIT media lab looks as futuristic as you’d expect it to. Its new six-story building near Kendall Square is all glass and metal, like something out of Avatar or Area 51. The World of Tomorrow feel extends inside, where roughly 150 elite graduate students work with a few dozen esteemed faculty and visiting scholars on some of the most cutting-edge technology in the world. On a recent afternoon, a group of students on one floor was tinkering with a robotic ankle. In the lobby, two scruffy guys swatted at a ball on a computerized ping-pong table amid animated fish projected onto the table’s surface. Upstairs, another student “painted” a television screen with virtual images rendered by strokes of a large wooden brush implanted with a digital eye.
Since its founding 25 years ago, the Media Lab’s goal has been to create high-tech inventions that have practical applications, a mission it has achieved with impressive frequency: The lab is responsible for such innovations as MPEG-4 digital audio files as well as the electronic ink in e-readers.
Yet among the fields in which the lab has been most influential is one that may be less obvious: music. And that phenomenon is largely owed to the influence of one man, MIT professor of music and media Tod Machover.
Later this year, in Monaco, Machover will debut an ambitious “opera of the future” he’s composing: Death and the Powers, which tells the story of a dying inventor who uploads his consciousness into a room full of intelligent objects, called The System. Though the opera has the familiar arrangement of strings and woodwinds, Machover and his team at MIT are also developing a host of new technologies specifically for the production – from life-size musical robots to gesture-controlled “disembodied performance” gear.
Such work isn’t as esoteric as it might seem. Machover uses his compositions to develop new ideas and experiment with new technology – innovations that have, over the past 25 years, filtered into everything from hospitals to your living room. And if you want to understand where technology will take the art form in the next decade, you need look only at what Machover and his band of scientists, graduates, gamers, and engineers are up to these days – and how they’re planning to transform the way we consume and develop music in our everyday lives.
The son of a juilliard-trained pianist and a computer-graphics pioneer, Machover first started thinking about technology and music as a kid growing up in the ’60s, when he decided to modify his cello to create the rock music he was hearing in his head. He started by rigging up a homemade amplification system, but soon he was composing with tape recorders and layers of tracks produced by his “electrified” cello.
After earning two degrees at Juilliard and a long residency in Paris, Machover joined MIT, where he launched his Hyperinstruments Group in 1986. At first he conceived of “hyperinstruments” as technological means through which musicians could augment their own performances to further unleash their
expressiveness. As his modifications to acoustic instruments (violins, bows, pianos) grew more complex, he began composing his own operas as a kind of workshop for his innovations. His first production, 1987’s VALIS, based on a novel by science-fiction writer Philip K. Dick, ran vocals through a synthesizer to create otherworldly effects. When the show debuted at the Centre Pompidou in Paris, it seemed more suited for fans of Kraftwerk than opera. “It was a scandal,” Machover recalls with a smile. “Half the audience would stand up and yell, and half would stand up and throw things. They got really angry. To them this was too populist.”
But to truly be populist, Machover realized, he needed hyper-instruments that put the power of expressiveness not only in the hands of virtuosos, but also in the hands of nonmusicians. So he composed a mini opera for magicians Penn & Teller using a so-called sensor chair that lets the sitter create music by waving one’s hands through fields of charged particles. He built the “Brain Opera,” now permanently installed at the Haus der Musik in Vienna, which offers a physical environment built from a collection of hyperinstruments. As people pass through the space, they create music by waving their hands in front of a wall or turning a steering wheel. He also created Toy Symphony, an educational project that, among other things, allows toddlers to “conduct” music by squeezing soft play toys. Yet another Machover innovation was Hyperscore, a composition program that creates music based on how the user draws images on a computer. Adam Boulanger, a Ph.D. student of Machover’s, has been using Hyperscore at Tewksbury State Hospital to enable patients with cerebral palsy, spina bifida, schizophrenia, and other conditions to make music. “The theme of everything we do here is accessibility,” Boulanger says. “We’re showing that anyone can compose or perform significant music if they have the right tool.”
Despite his innovations, Machover’s most lasting influence may be in the philosophy he passed on to two of his most accomplished former students: Alex Rigopulos and Eran Egozy. The pair met in the early ’90s while they were completing degrees at MIT through the Media Lab. There, they both ascribed to Machover’s gospel – and soon began looking for ways they, too, could bring musicianship to ordinary people. And what better means to engage a new generation, they figured, than with video games? In 1995 they formed their company, Harmonix, which stumbled for years before hitting it big with a series of interactive music-game franchises: Karaoke Revolution, Guitar Hero, and Rock Band. Today Harmonix is credited with creating a pop-culture phenomenon, and an industry with more than $2 billion in revenue. Still, if you ask Rigopulos and Egozy about the secret behind their success, they don’t hesitate to point back to their alma mater. “It gave us this brash, almost irresponsible sense of adventure to try something that had never been done before,” says Egozy. “We didn’t even think about being afraid of failure.”
On a warm night in Boston, music fans file into Symphony Hall to celebrate the 125th season of the Boston Pops. Conductor Keith Lockhart leads the audience into the past with selections from Brahms to Brubeck – a tribute to founding conductor Arthur Fiedler, who led the Pops from 1930 to 1979. Then Lockhart takes a surprising turn.
He starts innocently enough, with a sing-along of Beatles songs. Then, behind the orchestra, a 42-by-21-foot screen appears and shows an animated movie of the Fab Four. But it’s not a Yellow Submarine-era cartoon. It’s from a video game: The Beatles: Rock Band.
The game was, of course, developed by Harmonix, and the moment could easily be seen as the ultimate validation for the company – and for the ideals that guided its formation. Lockhart, for one, is pleased with the power of new music technologies to engage and empower new generations of listeners. “Anything that brings great music and great art forms to larger groups of people is worth exploring or experiencing,” he says.
But the moment is also a conspicuous reminder of what some skeptics find off-putting about the direction music and technology are headed. “One of the problems in virtual reality is that the line is blurred between actually being able to do something and virtually being able to do something,” says Lockhart, “and I find that a bit frightening.” (As if to acknowledge this concern, this fall Harmonix will release a new version of Rock Band that allows players to perform more authentically on real six-stringed guitars.)
Machover believes the music – not the technology – is the message. And if this means alienating the purists who think that all these gadgets are somehow inauthentic next to a violin or cello, so be it. “What’s authentic is anything that allows an individual to communicate,” he says.
He thinks the next 25 years will take music technology to a more intimate level. “We have just scratched the surface of how the expert-amateur boundary will be blurred with games like Guitar Hero. Over the next five years we will see an explosion of music created specifically to be mixed, mashed, and morphed by users, leading to personalized expression at an unprecedented level,” Machover says. “Most profoundly, we are on the cusp of understanding much better why music affects us at all, how it modulates our emotions. This will lead to ‘personal music,’ fine-tuned to an individual’s neurology, experience, and mood. I can’t wait to hear that.”
For Machover, the future of music is expression, not expertise. “Music is about conveying human emotion and ideas with other people,” he says, “and it doesn’t matter if you play a wrong note.”