How to Game the College Rankings
Northeastern University executed one of the most dramatic turnarounds in higher education. Its recipe for success? A single-minded focus on just one list.
In 1996, Richard Freeland looked across the sea of crumbling parking lots that was Northeastern University and saw an opportunity few others could. As the schoolâs new president, he had inherited a third-tier, blue-collar, commuter-based university whose defining campus feature was a collection of modest utilitarian buildings south of Huntington Avenue, with a sprinkling of newly planted trees.
The university had been a victim of many things, most notably federal cutbacksârolled out in the mid-â80sâthat had left many colleges scrambling for money to close their budget gaps. These cutbacks, combined with dwindling enrollment, had forced Northeasternâs previous president, Jack Curry, to slash the budget and cut 875 jobs in the early 1990s. When he announced the layoffs to his staff, Curry burst into tears. âTo say it was an institution in turmoil would be an understatement,â says a vice provost from that time.
But Freeland, the man who had helped successfully launch UMass Boston over the previous two decades, had a plan. Freeland believed that if Northeastern could justify its increased costs to students and parents, it could be saved. And one gauge consistently determined a collegeâs value: its position on the U.S. News & World Report âBest Collegesâ rankings. Freeland observed how schools ranked highly received increased visibility and prestige, stronger applicants, more alumni giving, and, most important, greater revenue potential. A low rank left a university scrambling for money. This single list, Freeland determined, had the power to make or break a school.
During his tenure, Curry had made improvements at Northeastern, but none of these changes could budge the schoolâs U.S. News ranking. Working with the mantra âSmaller but better,â Curry reduced class sizes and clamped down on admissions. He also tried to attract students from beyond Boston by creating a more welcoming campus, replacing some of the crumbling blacktops with new buildings, including a library and a recreation center. From the U.S. News perspective, these changes did little to influence the schoolâs reputation, the most statistically important metric in the ranking system. Curry left the school in 1996 at number 162.
Freeland swept into Northeastern with a brand-new mantra: recalibrate the school to climb up the ranks. âThereâs no question that the system invites gaming,â Freeland tells me. âWe made a systematic effort to influence [the outcome].â He directed university researchers to break the U.S. News code and replicate its formulas. He spoke about the rankings all the timeâin hallways and at board meetings, illustrating his points with charts. He spent his days trying to figure out how to get the biggest bump up the charts for his buck. He worked the goal into the schoolâs strategic plan. âWe had to get into the top 100,â Freeland says. âThat was a life-or-death matter for Northeastern.â
Founded in the 1930s and 1940s by David Lawrence as two separate newsweeklies, U.S. News and World Report merged in 1948, but it wasnât until 1983 that the publication printed its first cover story ranking Americaâs top 50 colleges. The issue happened to coincide with a sudden robust interest in higher education among the general population: Between 1970 and 1983, college enrollment increased 47 percent. What had once been considered a privilege for the wealthy or brilliant few was increasingly becoming the entry fee to the middle class. For the first time, a college degree was considered necessary, but how to choose among the thousands of institutions conferring degrees? Thus followed a new demand for unbiased, quantitative informationâjust as Consumer Reports rated washing machines, college rankings would serve as a first-time buyerâs guide to higher ed.
Along with the U.S. News list, the New York Times had just released Edward Fiskeâs first Guide to Colleges, and in 1984, the College Board began regularly selling SAT prep books. But none had the authority of U.S. News. Billionaire publisher Mort Zuckerman seized the moment and purchased the magazine, along with its rankings franchise, in 1984.
In the offices of U.S. News & World Report in Washington, DC, Robert Morse has labored for decades, crunching numbers for college rankings (this year, theyâll be released on September 9). He spends his days staring at two computer monitors, analyzing the data that schools submitted over the summer. For a man whose lifeâs work triggers a yearly cage match among universities, Morse is far from intimidating. He slouches and shuffles, letting the plastic dry-cleaner clips on his shirt go unnoticed. Yet as chief data strategist and developer of U.S. Newsâs secret rankings sauce, Morse has helped the magazine become one of the most feared and influential voices in the world of higher education.
When the U.S. News editors first devised a formula that declared, with statistical accuracy, which school was on top, they quantified something previously thought to be intangible. For generations, colleges and universities had generally relied on a mysterious brew of prestige and reputation. Suddenly, legacies and traditionâqualities that had taken decades, and sometimes centuries, for schools to cultivateâwere less important than cold, hard data. Schools that once relied on children of alumni and word of mouth were exposed by their own stats, including graduation and retention rates, admissions data (acceptance rate, average SAT score), academics (class size, number of full-time faculty), and reputation (peer reviews). Needless to say, U.S. Newsâs college rankings landed on the world of higher education with a thud.
With their authoritative tone, the rankings also introduced new possibilities. Before they appeared, it was doubtful that NU could ever rub shoulders with Boston College or Harvard. But now, with a codified system out there, nearly everything was reduced to numbers. And those numbers could be beat. âThey give you a playing field on which you can play,â Freeland says of the rankings. They give schools âa way to compete.â
From the start, schools have argued that the rankings are subjective. Defenders of the rankings maintain that the system exposes students to more schools and helps the consumer compare products. Regardless, students, graduate schools, and employers have embraced the list, giving it unprecedented power. An unintended result, however, is that schools need to spend more to stay competitive in the categories that U.S. News considers important. Universities may be in the business of education, but itâs a competitive business in which all compete for students and revenue. With an arms race to the top, higher education has soared out of reach for an increasing number of Americans. NU tuition alone in 1989 was $9,500; today itâs $42,534.
âYou can love us or hate us, but weâre not going away,â says U.S. News editor Brian Kelly. âUniversity officials realized weâre much more valuable to them than not.â He deflects criticism, saying, âItâs not up to us to solve problems. Weâre just putting data out there.â He does, however, admit that the rankings system can be gamed.
For those at Northeastern, breaking into the U.S. News top 100 was like landing a man on the moon, but Freeland was determined to try. Reverse-engineering the formulas took months; perfecting them took years. âWe could say, âWell, if we could move our graduation rates by X, this is how it would affect our standing,ââ Freeland says. âIt was very mathematical and very conscious and every year we would sit around and say, âOkay, well hereâs where we are, hereâs where we think we might be able to do next year, where will that place us?ââ
Figuring out how much Northeastern needed to adjust was one thing; actually doing it was another. Point by point, senior staff members tackled different criteria, always with an eye to U.S. Newsâs methodology. Freeland added faculty, for instance, to reduce class size. âWe did play other kinds of games,â he says. âYou get credit for the number of classes you have under 20 [students], so we lowered our caps on a lot of our classes to 19 just to make sure.â From 1996 to the 2003 edition (released in 2002), Northeastern rose 20 spots. (The title of each U.S. News âBest Collegesâ edition actually refers to the upcoming year.)
Admissions stats also played a big role in the rankings formula. In 2003, ranked at 127, Northeastern began accepting the online Common Application, making it easier for students to apply. The more applications NU could drum up, the more students they could turn away, thus making the school appear more selective. A year later, NU ranked 120. Since studies showed that students who lived on campus were more likely to stay enrolled, the school oversaw the construction of dormitories like those in West Villageâa $1 billion, seven-building complexâto improve retention and graduation rates. NU was lucky in this regardânot every urban school in the country had vast land, in the form of decrepit parking lots, on which to build a new, attractive campus.
There was one thing, however, that U.S. News weighted heavily that could not be fixed with numbers or formulas: the peer assessment. This would require some old-fashioned glad-handing. Freeland guessed that if there were 100 or so universities ahead of NU and if three people at each school were filling out the assessments, he and his team would have to influence some 300 people. âWe figured, âThatâs a manageable number, so weâre just gonna try to get to every one of them,ââ Freeland says. âEvery trip I took, every city I went to, every conference I went to, I made a point of making contact with any president who was in that national ranking.â Meanwhile, he put less effort into assessing other schools. âI did it based on what was in my head,â he says. âIt would have been much more honest just to not fill it out.â
With unwavering dedication to the rankings, Freeland jockeyed his school up 42 spots to 120 over eight years. Though impressive, getting to the elusive top 100 remained Freelandâs ultimate goal, but he had âgamedâ the system as far as he could on his own. To break into the top 100, heâd need more intel on the news magazineâs methodology. He would also need U.S. Newsâs complicity. âWe were trying to move the needle,â Freeland says, âand we felt there were a couple of ways in which the formula was not fair to Northeastern.â
And so it was in 2004 when Freeland, a 63-year-old with bushy gray eyebrows and slightly unkempt hair, stepped out of a taxi near the waterfront in Washington, DCâs fashionable Georgetown neighborhood. With his head down, his lips tightly pursed, he marched into the red-brick offices of U.S. News, determined to make the rankings wizard, data guru Robert Morse, his accomplice.
Meanwhile, other schools that couldnât successfully game the system were trying to cheat their way to the top.
In 2008, Baylor University told newly admitted students that theyâd receive a $300 campus-bookstore credit if they retook their SATs, and $1,000 a year in student aid if the scores improved by more than 50 points. In 2009, an administrator at Clemson University, whose president shared Freelandâs rankings fixation, admitted the school misrepresented financial information and purposefully rated institutions low on the peer assessments.
In 2011, Iona College officials admitted to misreporting acceptance rates, SAT scores, graduation rates, and alumni donation amounts over the course of a decade. In 2012, Claremont McKenna College copped to misreporting SAT scores for several years. Also in 2012, George Washington University admitted to inflating the percentage of students who graduated at the top of their high school classes, and Emory University said it had misreported high school GPAs for four years and SAT scores for nearly a dozen years.
Despite those instances, Brian Kelly, of U.S. News, says that misreporting is rare. U.S. News crosschecks some data with government records and other published sources, but that wonât help if a school is lying about data across the board. When misreporting does occur, U.S. News says it might temporarily âun-rankâ a school if the bad data affects its standing. âNinety-nine point nine percent of the schools are treating this seriously and reporting with integrity,â Kelly insists.
Cases of misreporting have only strengthened the anti-rankings movement, which has been around for decades. Lloyd Thacker, who worked in college admissions and founded the Education Conservancy, in Portland, Oregon, is one of U.S. Newsâs fiercest critics. He says research shows that the rankings distort the way students, parents, high schools, and colleges pursue and perceive education. âHave rankings contributed to anything beneficial in education?â he asks. âThereâs no evidence. Thereâs lots of evidence to the contrary.â
Because schools reap benefits from a high spot on U.S. Newsâs list, he says, it makes sense for them to continue to throw money at the metrics. From the mid-1990s to the mid-2000s, for instance, to lure students with high GPAs and SAT scores, private four-year schools increased spending on merit-based aid from $1.6 billion to $4.6 billion. Studies show that for every 10 merit-based scholarships, there are four fewer need-based scholarships. Thatâs because schools often base merit on test scores, and students from lower-income families generally donât test as well, largely because they spend less on tutors and SAT prep. Tuitions rise to help universities keep pace, further reducing middle-class access to a top-flight education.
Freeland landed in Morseâs U.S. News office that day in 2004 to discuss, among other things, how the publication handled students enrolled in NUâs co-op program. Long the cornerstone of the universityâs curriculum, the program let students take breaks from their studies in order to gain professional experience in their chosen fields for months at a time. NU maintained that this jobs program was critical to its graduatesâ successâit was one of the schoolâs greatest strengths. Unfortunately, Morse counted co-op students in enrollment data, making it look like more students were using university resources than actually were. This brought down the U.S. News âfinancial resourcesâ criterion, thereby hurting NUâs rank. From 2002 to 2003, that was the only metric in which NU actually did worse.
Though Morse initially wouldnât go along with Freelandâs request to change the magazineâs methodology to address Northeasternâs co-op, Freeland emerged from the U.S. News office more confident than ever that NU could crack the top 100. Morse had agreed with Freelandâs overall reasoning and helped him better understand the criteria. It was just enough insight for Freeland to work with.
The following year, NUâs ranking advanced to 115. With its rankings that fall, U.S. News published a three-page article praising Northeasternâs co-op program. The publication wouldnât reveal whether it changed its policy, but Northeastern no longer includes co-op students in its reporting. At that point, Freeland says, âWe discovered that we could actually start to move pretty quickly. The more we discovered that, the more we focused on it.â
Not everyone, however, was onboard with the rankings push. âI think one of my tasks actually at Northeastern was to persuade the faculty that this was a good and reasonable vision,â says Ahmed Abdelal, provost at the time. But, he says, âIâm not going to claim that we converted every faculty member.â One former administrator describes NUâs former president as controlling and single-minded. âI think Freeland was much more focused on the ratings and not necessarily on what it took to improve the quality of the institution,â the former administrator says. âHe was going to do whatever it took to increase the ratings.â
The day after Freeland retired in August 2006, he was vacationing on Marthaâs Vineyard when he heard the news: The U.S. News rankings were out, and Northeastern had broken through the top 100, all the way to number 98. In his decade as president, Freeland had lifted the school up more than 60 spots, prompting the Boston Business Journal to later call NUâs rise âone of the most dramatic since U.S. News began ranking schools.â Before Freeland left, the trustees had voted again on an amount for his retirement supplement, awarding him around $2 million to acknowledge his success. Years later, the state would name him commissioner of higher education.
Leery of being misunderstood, Freeland tells me, âIt may have seemed a little foolish at the time or a little shallow at the time, but the proof of the pudding is in the eating.â
From his office, current Northeastern president Joseph Aoun looks out over the construction of a new residence hallâthe schoolâs 12th since 1996âsituated behind the YMCA where NU began more than a century ago. Wearing a suit, a pink shirt, and a salmon-colored tie, he sits cross-legged in a white leather chair and holds his eyeglasses between his thumb and index finger. When he requests water from his secretary, it arrives in a stemmed glass with two lemon slices. This is a man who is clearly used to winning, and takes pleasure in the spoils.
When Freeland retired in 2006, NU trustees saw in Aoun the right man to keep the rankings machine rolling. Aoun had been dean of the College of Letters, Arts and Sciences at the University of Southern California; during his tenure as dean, the school rose eight spots between 2000 and 2006, breaking into the top 30. Aoun says that NUâs trustees were targeting candidates from large private schools that had âmoved up in the trajectory.â
In many ways, Aoun tries to distance himself from Freeland. He resists talking about the schoolâs meteoric rise over 17 yearsâfrom 162 to 49 in 2013âand plays down the rankings, brushing them aside like an embarrassment or a youthful mistake. âThe focus on the ranking is not a strategy, for a simple reason,â he says. âYou have thousands of rankings. So you will lose sleep if you start chasing all of them.â While itâs true that U.S. News no longer appears in the universityâs strategic plan, it does appear in NUâs portrayal of itself: The school has no qualms using its high ranking in recruiting materials and publicity videos. Yet multiple Northeastern administrators expressed concern over this articleâs focus on the rankings. One vice president telephoned Bostonâs editors in a panic.
Despite Aounâs carefully crafted image, the schoolâs actions undercut his words, as gaming U.S. News is now clearly part of the universityâs DNA. And Aoun is a willing participant. âHe may not admit to it, but thatâs definitely whatâs going on,â says Bob Lowndes, who is retiring as vice provost for global relations. Ahmed Abdelal, provost under both Freeland and Aoun, says the two presidents have shared âthe same goal: further advancement in national ranking.â
Under Aoun, in an apparent effort to score rankings points by lowering the percentage of accepted students, NUâs admissions department received a mandate: Increase applications at home and abroad. Northeasternâs new science and engineering complex, colorful Adirondack chairs throughout campus, and star faculty members like Michael Dukakis are all intended to advance reputation and lure top students. âThey poured a ton of money into admission recruiting,â says a former NU admissions officer. âWe had amazing amounts of latitude to travel overseas. We worked really hard, and we traveled like crazy.â
There were other tricks, as well. In 2009, NU stopped requiring SAT scores from students attending international high schools. By removing a barrier to foreign students, who typically score lower if they take the SATs at all, NU boosted its application numbers without jeopardizing its overall testing average. Those foreign students, ineligible for federal aid, also tend to pay full freight. Since 2006, the percentage of international undergraduates has jumped from just under 5 to nearly 17. In the 2012 to 2013 school year, NU had the most international students of any university in New England, and the 10th most in the country.
Aoun also began using spring enrollment to his advantage. In 2007 the school introduced N.U.in, a program that invites students with lower grades and SAT scores to spend their first semester abroad and begin their on-campus experience in the spring. U.S. News does not collect data for spring entrants, so those studentsâ lower grades and scores are excluded from the rankings. Editor Brian Kelly explains that U.S. News doesnât require spring data because the federal government doesnât either, but he concedes, âItâs possible that is a gaming window.â
Aounâs tactics seem to have worked. Last year, Northeastern received its highest number of applications, almost 50,000 for 2,800 spots. Thatâs nearly five times more than in 1990. Enrolled students were more qualified than ever before, with average SAT scores up 22 points from the previous year. âTheyâre at the level of Ivy League schools,â says Henry Nasella, NUâs chairman of the board of trustees. âPeople are always talking about the value of their degree,â says last yearâs student-body president, Nick Naraghi. âNot only am I getting what I was thinking I would get, Iâm getting much more.â
Serving as the president of a top-50 school has its perks. Aoun lives in a 14-room townhouse overlooking Boston Common that the school purchased his first year, now worth $8.9 million. A chauffeur drives him to campus. Aounâs salary increases each year, and in 2011, he was the second-highest-paid college president in the country, at $3.1 million, though that included a $2 million retirement supplement. (Harvardâs president, Drew Gilpin Faust, came in 54th that year, with total compensation just under $900,000.) In addition to Aoun, seven Northeastern employees made over half a million dollars that year.
Northeastern has essentially created a blueprint for any school that can stomach following it, but gaming the U.S. News rankings has its costs: According to a study published this year by Research in Higher Education, a mid-30 school like the University of Rochester would have to spend an extra $112 million on faculty salary and student resources alone to rise a total of two spots. To feed the beast, Aoun has tapped several new streams of revenue, including satellite graduate campuses in Charlotte, North Carolina, and Seattle, as well as online education. Nasella calls them âvery importantâ and claims they are âvery profitableâ for the school.
âThereâs no rankings problem that money canât solve,â says Michael Bastedo, director of the Center for the Study of Higher and Postsecondary Education. Actually, there may be oneâa ranking system with enough credibility to overshadow all others has appeared on the horizon, and this one will be much tougher to game. Concerned about the skyrocketing costs of a college education, the White House is launching a federal rating system that, unlike U.S. Newsâs, will focus less on student inputs (e.g., SAT scores) and more on the resultant value of a degreeâweighing a schoolâs tuition costs, graduation rates, and postgraduate salaries. The government hopes to implement the system this fall. Donât expect to find NU at the top: In Money magazineâs 2014 âBest Colleges for Your Money,â which uses a similar methodology, Northeastern landed at the bottom of the third quartile, at number 433.
By the Numbers
NORTHEASTERNâS RISE IN THE U.S. NEWS COLLEGE RANKINGS
Check out all of our Best Schools 2014 coverage.
Source URL: http://www.bostonmagazine.com/news/article/2014/08/26/how-northeastern-gamed-the-college-rankings/