Questionable Science Behind Academic Rankings | Hacker News Reader

Questionable Science Behind Academic Rankings

nytimes.com

27 pointsJ3L240416 comments

While these publications try to rank universities, I engage in the smaller task of trying to rank undergraduate colleges: http://college.mychances.net/college-rankings.php

The decision to rank them based on students' decisions of where to attend may be questionable, but at least the "science" is straightforward.

Seems like a nice system, my only question is how do you get this data? Just students self-reporting?

Thank you. Yes, this is based on student self-reporting. The overall reporting quality appears to be quite good. The overall feedback rate (% of students who tell us their acceptances/rejections after the admissions cycle is over) is lower than I'd like, but higher than I'd expect.

It seems like most of the things that make a university "good" are things that a prospective undergrad, with no college experience, would be least likely to know.

I'm not implying that the current system is a a good one.

I agree, and this is why I distinguish college, which is a place that undergrads go for 4 years, from university, which is the larger institution as a whole.

For certain characteristics of colleges, I believe it is fair to see students and parents as informed consumers; e.g., when considering cost and prestige, they probably do a good job. Without anything to substantiate this, I would also guess that students do a reasonably good job of selecting institutions that will make them employable (we're getting into fuzzy territory so I don't want to make too strong of a claim).

But I agree that this method of ranking colleges would not translate into a ranking of universities as greater entities. If Kenyon suddenly made tuition free for everyone, they might lure away enough students to become the "Best" in my rankings, but it would not therefore follow that Kenyon's academic output trumps that of Princeton.

I think GP was referring to the undergrad program. For what it's worth, I haven't met many students who knew much about their degree program's curricula or their general education requirements. Not knowing this makes them seem ineffective at evaluating the college. Given all the stories I hear of, for example, art students surprised that they cannot find jobs, I'm not sure I trust incoming students as estimators of employability either. The only things I really would trust them to evaluate are the school's reputation and the first year's tuition.

To a large degree, I agree with both you and RK.

I agree with what you said, although I actually meant that students themselves probably don't know what makes a college good for them, education-wise and in the long run.

I really don't know the answer either. Being surrounded by other smart students, class size, specific teachers, research opportunities, campus culture, student body diversity, etc?

I think the biggest thing about going to college is that it's totally different than what you've experienced before. A prospective student will necessarily be deciding on very incomplete information. Unfortunately, you probably need about a semester's worth of experience before knowing if a given school were truly a good fit...

Is there a section where I, as a college student, can just add my statistics?

Yes, definitely. It's a 2 step process that probably has way too much friction right now (and at some point soon I'll be asking HN for feedback, but that's not for right now). (1) Create for an account ( http://www.mychances.net/signup.php ). Once you confirm, it takes you into step (2) automatically, creating your profile and then letting you add the colleges that you got decisions from.

Great. I thought my statistics may be interesting* but I didn't really see a signup area for college students vs prospective college students.

* For Northwestern, my GPA was very low while my test scores were probably abnormally high.

My last startup was in higher ed, and the stories about manipulating ranking are amazing. Everybody knows what goes into the rankings, and everybody manipulates the inputs.

As an example, one of the inputs is percentage of alumni who donate. So one large school has a gigantic push every year to get just one dollar from every alumni that they can. I actually sat in on a seminar where somebody from the university broke down all the bizarre things they were doing with admission, freshman retention, alumni relations, and so-on, just to game US News and World Reports.

Princeton is obsessed with getting $1 from me every year for this reason.

The rankings are not completely worthless, but they certainly are overrated. But you ask for them.

University ranking is a strange beast. On one hand, it may merely indicate the amount of endowment a university receives from its alumni per capita. On the other, it may truly rank a school by the merits of its research.

But more often than not, I find that university ranking is mostly to measure how endowed a university is, no pun intended. If a university--particularly a well-established research university--were so good, why couldn't it turn a mediocre student into a star student?

(Went to Arizona State; not exactly high on the US rankings)

How about another metrics that measure the number of billionaires where they originally dropped out from? That would be interesting.

Reply on news.ycombinator.com