[转]Turning the Tables: Ranking the MBA Rankings

希望

by John A. Byrne

Forbes says the full-time MBA program at the University of Stanford is best. BusinessWeek say it’s the University of Chicago. U.S. News & World Report has yet another answer: Harvard Business School and Stanford in a dead tie. Say what you will about the lack of consensus on what constitutes the best MBA program in the U.S., but at least three different areas of the country—east, west and mid-west—can claim a No. 1 business school. Meantime, British-based publications seem to prefer MBA-granting institutions not based in the U.S. at all. The Economist claims the best school in the world is IESE in Spain, while the Financial Times begs to differ, preferring London Business School in its backyard. When The Wall Street Journal did an annual ranking, it often put Dartmouth’s Tuck School at the top.

Or just take the ranking of any prominent school at random and puzzle over the disparities among them: One of the finest business institutions in the world is the University of California’s Anderson School in Los Angeles. Yet, The Economist ranks it 50th behind Boston University. BusinessWeek ranks UCLA’s business school 14th, U.S. News & World Report gives it a 15th rank, and the Financial Times puts it at 33.

How is it possible that the five major rankings of business schools all have different winners at the top? How can they rank a single school, such as UCLA, so differently? Chalk it up to the differing ways each publication cranks out its list of the best.  The different methodologies employed for these rankings have as much if not more to do with where the schools place rather than the actual quality of the institution or the MBA experience.

The general problem with most rankings is that they measure what is easy to count, largely statistics on average GMATs and GPAs, starting salaries and bonuses, rather than what really matters. The latter is harder to get at, but essentially what is more important is whether you think the cost of the degree is worth the time and effort. Was the teaching quality superb? Did you learn as much from your fellow students as you did from the faculty? Did the coursework prepare you for a successful career? Do you feel the alumni network will help you throughout your professional life? Did you create friends who you will keep for the rest of your life? Did the degree enrich you far beyond the monetary rewards? Those are questions that few rankings tend to measure, yet those are the important questions.

By searching the web, you’ll find lots of other rankings besides these five major rankings. (The Wall Street Journal stopped ranking full-time MBA programs in 2007.) Ignore them. Often, they’re put together by people with an axe to grind against the big five or by others who are trying to sell you something. The theory is simple: If you don’t like these results, you put together your own ranking. That’s been done by faculty who want to reward academic research over teaching quality and MBA salaries, and that’s been done by consultants or others who are clamoring for attention to sell you something. As imperfect and flawed as the five major rankings are, they represent legitimate and good faith efforts by conscientious journalists who cover higher education day in and day out. They are looking for what they perceive to be the best ways to measure the quality of a business school education.

Which leads us to an inevitable question: How would you rank the rankings? First, an admission: I created BusinessWeek’s ranking 22 years ago. At the time, the BusinessWeek list was the first major MBA ranking and its approach was unique. BusinessWeek ranked the top MBA programs by surveying recent graduates and corporate recruiters. We asked the questions that had rarely been asked before. A few examples: “Did the professors convey enthusiasm for what they taught?” “In classes taken with popular or distinguished professors, how would you rank their accessibility after class?” ”Were you given a way of thinking or approaching problems that will serve you over the long haul?” “Did you have the feeling that your teachers were at the leading edge of knowledge in their fields?” “What percentage of your classmates would you have liked to have as friends?” These are questions that get far closer to the quality of the MBA experience than a median GMAT score for an accepted applicant. Until the BusinessWeek survey debuted, rankings relied primarily on the opinions of B-school deans, faculty, or top executives who were asked to name the top programs—even though they often had only indirect knowledge of many of the schools—or merely ranked schools on available statistical data.

That first BusinessWeek ranking in 1988 upset the traditional order of things. Northwestern University’s Kellogg School beat out both Harvard and Stanford. Dartmouth scored high, taking sixth place. Yale, often in previous Top 20 lists, failed to make the ranking because of dramatically poor grades from recent graduates. Over the past 22 years, through 11 separate rankings by BusinessWeek, Kellogg has claimed the number one spot the most, five times; Wharton is second with four No. 1 rankings by BusinessWeek, and Chicago’s Booth School of Business is next, having won the top spot twice, in the last two consecutive polls.

Many years have obviously passed since I had any involvement in the rankings and I left BusinessWeek’s employ at the end of 2009. Through all those years, however, I’ve maintained a curious interest in the world of business education. I’ve visited the campuses of well over 100 business schools. I’ve interviewed hundreds of deans, faculty, and students. I’ve also studied the core competition. Here’s my two cents on how the best stack up against each other.

1. BusinessWeek. The granddaddy of the MBA rankings, BusinessWeek’s Best-B Schools list made its debut in 1988 with what was then a radical approach to judging the quality of a school: pure customer satisfaction. It was the first to systematically survey recent MBA grads and ask them about their satisfaction levels with their educational experience and the first to ask corporate recruiters their opinions of the MBAs they interview and hire. To adjust for possible one-year bias, BusinessWeek includes in its ranking the two previous MBA grad and recruiter surveys. Why is it still the best? Because it puts otherwise unavailable information in the marketplace to help people with a very important career decision. Applicants can weigh the commonly available information—from average GMAT schools to median salaries—on their own, without the subjective weighting of a journalist. The BusinessWeek ranking is crystal clear in what it measures: How well the schools serve their two primary markets, students and their ultimate employers. Over the 11 rankings published by BusinessWeek in the past 22 years, only three schools have been at number one: Kellogg topped the list five times, Wharton did it four times, and Chicago has been number one twice.

In recent years, BusinessWeek has since added an intellectual capital measurement, which gets 10% of the overall weight in the ranking. This latter survey examines the contributions of faculty in 20 academic journals as well as book reviews of faculty books in The Wall Street Journal, The New York Times, and BusinessWeek. The ranking is published every other year, in October, in even-numbered years. So the next list will be published in October of 2010. BusinessWeek ranks the top 30 business schools and then lists without a numerical ranking another 15 “second tier” schools. The latest top five in the lineup:

1.University of Chicago (Booth)
2.Harvard Business School
3.Northwestern University (Kellogg)
4.University of Pennsylvania (Wharton)
5.University of Michigan (Ross)

Pro: The ranking does not rely on less reliable data from schools which are under pressure to report information in the best positive light. It does not include such factors as GMAT scores, undergraduate GPAs, or starting salaries, but makes the case that discriminating graduate students and the companies that hire recruiters are in the best possible position to judge the quality of a school. Its survey of MBA recruiters also is the most credible in the marketplace, largely because BW invests a great deal of time in finding the one person in every organization that hires business school grads who is responsible for MBA recruitment. BW also weights the opinions of recruiters based on how many MBAs they hire—a process that takes into account the valuable experience of the companies in the MBA market. Moreover, BusinessWeek does not try to stretch the data it gathers and rank more than 30 schools. Instead, it creates a “second-tier” group of schools, acknowledging that differences among those schools are statistically inconsequential.

Con: It’s an entirely U.S.-centric list that excludes some very good schools outside the U.S. (BW publishes a separate list of the top ten international schools using the same methodology). It’s also possible that a graduating class of MBAs, knowing that their answers will affect the ranking of their school, may be encouraged to report more positively on their experience than is warranted. The upshot: very small differences in MBA opinion could affect a school’s standing. BusinessWeek clearly knows this is a possibility because the magazine employs statisticians who “use a series of statistical analyses to test the responses for patterns that have a low probability of occurring if the students are answering the questions honestly.” If foul play is suspected, the surveys are tossed out of the calculation. There are many indicators of quality, such as GMAT scores of incoming students and post-MBA compensation that are not included in this ranking.

2. Forbes. The magazine has done six biennial lists based on a simple but important metric: the return on investment MBA grads have achieved after five years in the market. Stanford tops the latest ranking because the median salary of an MBA from the Class of 2004 was $225,000, highest among U.S. rivals. Forbes figures that the typical Stanford student from that class paid $235,000 to get the degree, a sum including two years of forgone compensation as well as tuition and fees. What’s ideal about the Forbes methodology is that it, like BusinessWeek, is putting information into the MBA marketplace that would otherwise be unavailable to applicants making important decisions. And let’s face it: the vast majority of people who get an MBA do so for the financial rewards that come from the degree. They seek it to get out of the pile of people competing for better jobs, faster and more important promotions, and greater compensation. If ROI turns you on, this is the ranking for you. The latest top five of this list of 75 ranked schools came out in August, 2009:

1.Stanford University
2.Dartmouth College (Tuck)
3.Harvard Business School
4.University of Chicago (Booth)
5.University of Pennsylvania (Wharton)

Pro: It’s simple and elegant and doesn’t pretend to measure the quality of the MBA experience or the actual education you receive. It’s all about dollars and cents, measuring the worth of a degree five years after graduation. It’s also based not on data supplied by business schools, some of which would be tempted to fudge the information to get a better ranking, but from Forbes own survey of 17,000 alums at 103 schools. The last survey for the Class of 2004 had an impressive response rate of 24%.

Con: Forbes ranking is U.S.-centric, though Forbes does separate lists of non-U.S. institutions that award MBAs in one year and in two-year programs. The biggest problem is that the ranking only measures compensation versus the cost of getting the MBA. Return on investment, while interesting and important, is hardly a measure of the quality of the education or the experience you’ll have.

3. U.S. News & World Report. The magazine ranks U.S. schools every year, using a vast amount of information and data that ultimately pots Harvard Business School in the first spot. The methodology takes into account its own survey of b-school deans and MBA directors (25% of the score), corporate recruiters (15%), starting salaries and bonuses (14%), employment rates at and shortly after graduation (14% to 7%), student GMATs (about 16%), undergrad GPAs (about 8%), and the percentage of applicants who are accepted to a school (a little over 1%). If you want a ranking that pretty much includes all the obvious factors of quality, this is your best bet. In it’s latest survey, released in mid-April of 2010, U.S. News ranked 97 schools. The latest top five in the 2010 survey has Harvard and Stanford tied as number one and Chicago and Wharton tied at number five:

1.Harvard Business School
2.Stanford University
3.MIT (Sloan)
4.Northwestern University (Kellogg)
5.University of Chicago (Booth)
6.University of Pennsylvania (Wharton)

Pro: U.S. News puts a good deal of effort into the ranking, cranks it out on an annual basis, and measures a lot of different factors that are clear indicators of quality. There are a lot of ties in its ranking of schools (four schools, for example, are tied at rank 33, while another four are tied at rank 40). That’s to be applauded because U.S. News is acknowledging that the differences are so small among these schools that it would be intellectually dishonest to say one is better than the other.

Con: About 60% of the ranking is based on data supplied by schools who therefore have plenty of reason to present the information in the best possible light to get a better ranking. There’s no way to independently check or audit the data provided by business schools. The ranking is completely U.S.-centric when there are many non-U.S. schools with better programs not listed at all. Finally, some of the ingredients in this ranking stew seem a bit silly: Why even bother to include the percentage of applicants accepted when you weight it at little more than one percent?

4. Financial Times. This 2010 ranking is the 12th year the Financial Times has ranked the best MBA schools. The winner: London Business School. The ranking is based on data collected from business schools and from alums three years after graduation. This survey includes responses from 8,001 alums of the Class of 2006 those most of the results are combined with two previous class surveys. The ranking also includes a research component, weighted at 10%, counting papers written by the faculty of each school in 40 academic journals during the past three years. For my money, it’s the best all-inclusive global ranking of the top 100 business schools, far better than The Economist list.

1.London Business School
2.University of Pennsylvania (Wharton)
3.Harvard Business School
4.Stanford University
5.Insead in France and Singapore

Pro: Arguably the single best global ranking of business schools which gives you a very good read on what the best MBA-granting institutions are around the world.

Con: The Financial Times is trying so hard to be global that it ranks many non-U.S. and European schools far too highly over institutions that have significantly better MBA programs. It’s just not even credible to think that London Business School can give you a better MBA education than Harvard, Stanford, Wharton, or Northwestern. The FT ranking also puts far too much weight (40%) on how much alums of these schools make three years after graduation. MBA compensation, after all, has more to do with what fields grads go into (investment banking and consulting usually pay the most) than the quality of the education. Insiders also believe this data, reported to the FT by the schools, is being fudged by some schools who also include in the number their Executive MBA grads who already have high paid jobs. For example, the reported “weighted salary” for an MBA from The Indian School of Business is $141,291, a sum that exceeds what MBAs are making, at least according to the FT, at Oxford, Cambridge, Northwestern, Berkeley, Duke, Michigan, and New York University, among many of the other top schools in the world. It simply strains credulity.

The FT also includes in its ranking other factors that have no impact on quality, such as the percentage of women faculty at a school. It may be politically correct to toss a factor like that into the mix, but you can’t presume it means the MBA program is less valuable because there are fewer female instructors on the faculty. The FT’s own methodology fails to clearly explain all the data it uses to rank schools so there’s a bit of a black box game. It notes, for example, that “eleven of the ranking criteria are based on data” from business schools. But it only mentions four of these 11 criteria, such as percentage of grads employed three months after graduation and an unexplained “FT doctoral rank,” whatever that means. The heavy reliance on unaudited data from schools also is suspect because some schools are likely to report information in the best possible light to gain a higher ranking.

5. The Economist. This is probably the most flawed, if not downright silly, of all the MBA rankings cranked out by a major media brand. The magazine is now up to its eighth annual ranking of full-time MBA programs. The ranking is based 20% on student and alumni surveys and 80% on data provided by the schools. There are some incredibly peculiar results in this ranking, which raise significant credibility issues. Pretty much no one in business education would agree that IESE is the best business school in the world or that Berkeley is better than Harvard, Dartmouth or Stanford. Or consider The Economist’s ranking for the University of California at Los Angeles. The Economist ranks this school 50th behind Boston University, University of Texas at Austin, and Georgetown University. Yet, BusinessWeek and U.S. News & World Report rank UCLA’s business school at 14th and the Financial Times puts it at 33. The Economist list ranks 100 top schools.

1.IESE Business School in Spain
2.IMD in Switzerland
3.University of California—Berkeley (Haas)
4.University of Chicago (Booth)
5.Harvard Business School

Pro: Takes a global perspective on business school education.

Con: The odd results of this ranking raise meaningful credibility issues with the methodology and the accuracy of the data some of these schools are providing to The Economist. Because 80% of the ranking is based on unaudited information from business schools, there’s a high likelihood that some data has been fudged. The Economist also throws into its ranking formula criteria that has little to do with the quality of education, such as the percentages of international and female students (giving these two questions alone nearly a 17% of the weight in the ranking), the range of overseas exchange programs (a 6.25% weight), and the number of languages offered (also given a 6.25% weight). That latter would be something more appropriate to undergraduate education.

6. The Wall Street Journal. This prominent business newspaper ranked full-time MBA programs for seven years, but stopped regularly doing an annual ranking with the September, 2007, list, probably because its flawed methodology led to quirky results and significant criticism. We’re ranking it nonetheless because many schools (especially those faring quite well on the list) still include this ranking in their marketing materials. The Journal ranking was conducted by pollster Harris Interactive which surveyed recruiters on 21 different attributes, including students’ leadership potential and strategic thinking, their previous work experience, the faculty and the curriculum, and the career services office.

Pro: It’s simple to understand, measuring only the views of the recruiters who come to campus to hire MBAs.

Con: The Journal got its list of recruiters to survey from the business schools, which provided the newspaper with the names of the actual recruiters, who showed up on campus. Most major MBA recruiters, however, send alums back to their schools to interview candidates. If you merely survey the recruiters who visit a particular school, you’re surveying entirely biased people. Would a Wharton MBA who recruits only Wharton students rate his or her school poorly? Not likely. In fact, what happened is that single companies that recruit MBAs would get numerous votes in the Journal survey—each from the alum of the school they recruit from. The result: this was less a ranking of the best business schools and more a ranking of the loyalty of a school’s graduates who happen to recruit MBAs at their alma maters.  Dartmouth’s Tuck School placed first in this survey four times largely because its small graduating classes and intimate educational setting make its alums more loyal than those from Harvard, which shockingly ranked 14th, or Stanford, which just as shockingly ranked 19th.  In the years that the Journal did this study, it would tell readers that it surveyed more than 2,000 recruiters. Problem is, there’s less than 300 companies in the world that actively recruit MBAs from enough schools to make a legitimate judgment about the quality of the institutions and the MBAs they produce.

BusinessWeek, in contrast, surveys far fewer corporate recruiters because it spends months finding out who are in charge of MBA recruitment and development at all the companies, which make the market for these graduates. Only then does BusinessWeek survey this group, getting the opinions of the top executive for MBA hiring in each organization—not alums that revisit their schools to hire students. Thank goodness the Journal had the sense to drop out of the ranking game because this one was so flawed it lacked all credibility. The top five in the last full-time MBA ranking by the Wall Street Journal:

1.Dartmouth College (Tuck)
2.University of California—Berkeley (Haas)
3.Columbia University
4.Carnegie-Mellon University
5.Massachusetts Institute of Technology (Sloan)

supernewbie

Stupid UK media have a reputation of self-reporting bias, ranking their garbage schools as the top of the world.

alvarosun

樓主你在寫畢業論文啊