A Stranger In the House of Ayer

As Jack Doorly’s stature within the Essex Street office grew in the 1990s, his spending habits began to match those of a man who had made his way in business. An avid golfer with a single-digit handicap, he spent his weekends on the links. He and his wife eventually bought more than five acres in Topsfield and built themselves a million-dollar dream house. In its garage they parked a Jaguar and two Cadillac SUVs.

Still, Doorly’s frustration with the conservative management of the fortune continued to simmer. And with the power he was amassing in the office, he saw opportunity to take increasingly large gambles that he hoped might pay off big for the family. He sank $2 million in a pair of dot-com companies that went bust, and invested close to $6 million in a Georgia turf company—it was rumored the owner had a membership at the Augusta National Golf Club, where Doorly desperately wanted to play—setting himself up as chairman. When his investments failed, he subdivided his losses into smaller amounts and distributed them among several family trusts. And, according to the suit against Doorly, when he won on a stock, he parceled out most of the gains among his personal accounts. One such successful bet was on a tech company called Saflink, which produces security products, like fingerprint readers, that help companies verify the identity and monitor the activity of their employees. That move netted a $4.5 million windfall.

By early 2002, Doorly knew the family had about $60 million in cash sitting in bank accounts drawing a paltry interest rate. He was sure he could put the money to better use and drafted a memo suggesting an investment fund for things like mortgages to family members who could borrow against their own chunk of the Ayer fortune. The idea was low-risk and low-profile enough to appeal to Caleb Loring, who agreed to put about $18 million into the fund and turn it over to Doorly as his pet project. Doorly eventually grew its value to $28 million. But along the way, the family alleges, he quietly helped himself to $15.5 million. Of that money, Doorly handed out millions of dollars in mortgages and business loans, lending to his lawyer, his golf partner, his personal trainer, and relatives. When terms were especially plum, or the paperwork especially thin, Doorly told himself that his benevolence was no different than what the Ayers showed their own family.

Doorly had demonstrated his aptitude for real estate investing with the three assisted-living facilities. So when he suggested developing a fourth in Durham, New Hampshire, Loring agreed to allocate $6.1 million to buy the land. But Loring says in court documents that what Doorly didn’t explain was that Orbit Construction (which the Ayers had no idea they’d launched) had bought the property in Durham 10 months earlier using money Doorly had withdrawn from family accounts. Doorly then allegedly used nearly $3 million of Loring’s cash to balance out the accounts he had previously drawn from—and when he happily reported to the Ayer family that the facility was finished, he left out the fact that he retained the title. It was a strategy the family claims he’d employ to similar effect more than once, buying out their interest in the Leominster strip mall and the New Hampshire car dealership by using their own money.

With so much in motion, Doorly was working 70-hour weeks, and the stress was catching up to him. In 2002, he underwent quadruple bypass surgery, which kept him out of the office for several months. The health scare forced Doorly to reevaluate his priorities. He apparently resolved to enjoy life more.

Doorly already belonged to at least three country clubs, including the TPC of Boston, which costs about $60,000 to join. An acquaintance says he tried to become a member of the North Shore’s Myopia club, a Brahmin stronghold, only to be turned away. He decided to spend his money elsewhere. Doorly started a holding company and used it to make his most audacious purchase: a 22-seat Gulfstream jet, for about $3 million. Borrowing the initials of his son, Adam Paul, he had “AP Enterprises” stenciled on the tail. The jet was ostensibly bought for his newest Florida company—another sports marketing firm, this one named All Access—but Doorly made frequent use of it to indulge his taste for gambling in Las Vegas and golf in Florida. Adam, now installed at All Access, seemed especially smitten with the kind of lifestyle the jet represented. When Doorly’s partner in the Florida venture, Peter Falcone, angled to use the plane for a trip to New York, Adam drafted an angry e-mail. “You gotta love this Falcone,” he wrote his father. “This guy is just trying to be us. He is not and never will be. I think we should strongly consider shutting this plane down to all employees of any company we are involved with.”

By early 2004, Doorly was keeping secrets from his own family. He began giving money to Sarah Hunt, who’d worked as a secretary in the Ayer office back in Doorly’s early years there and was now an administrative assistant at a hospital. She was going through a divorce and was having trouble making ends meet, and Doorly knew she wanted to pitch in for her daughter’s upcoming wedding. Over the next two years, he sent Hunt about 50 checks in amounts varying from $2,000 to $12,500, recording many of them in his desk ledger under coded entries like “Hunt Construction.” In all, Hunt received more than $300,000. The relationship eventually turned romantic. When they e-mailed, she took to referencing Doorly’s generosity, signing her notes “Mrs. Benjamin.”

Go on to the next page...