This will merely reflect reputation. How many people in academia have been at Harvard long enough to have an actual opinion?
Question: if China built the best university in the world today, how long would it take to reach number 1 on the list? I think decades. So much for having a yearly list.
I looked for details, from http://www.thes.co.uk/worldrankings/story.aspx . It requires registration, but here's a part of the method section:
"Our rankings contain two strands of peer review. The more important is academic opinion, worth 40 per cent of the total score available in the rankings. The opinions are gathered, like the rest of the rankings data, by our partners QS Quacquarelli Symonds (www.topuniversities.com) which has built up a database of e-mail addresses of active academics across the world. They are invited to tell QS what area of academic life they come from, choosing from science, biomedicine, technology, social science or the arts and humanities. They are then asked to list up to 30 universities that they regard as the leaders in the academic field they know about, and in 2007 we have strengthened our measures to prevent anyone voting for his or her own institution.
This year we have the opinions of 5,101 experts, of whom 41 per cent are in Europe, the Middle East and Africa, 30 per cent in the Americas, and 29 per cent in the Asia-Pacific region."
The rest of the score is split:
-10% which university do recruiters favor
-20% publication count
-20% student stuff, like teacher-to-student ratio (the only example they give)
-10% I couldn't find.
So there's definitely two selection effects: how did they collect their addresses, and who decided to answer a Times Online survey. I'm pretty sure my advisor wouldn't do it. Too busy.
Furthermore, they don't give the detailed methodology, and they admit to changing it since the first published results.
Overall, it seems like an honest effort, but what's the point? Would you advise a young person to make a decision based on this ranking?