Hacker News Reader

Comment by impendia | Hacker News Reader

I am a proud alumnus of Rice University and I see that it is on page 64, fifth from the bottom. Also, even more surprisingly, very near the bottom is Caltech.

This is obviously wrong. If their methodology says that Caltech is in the bottom 2% of US colleges, then one concludes that their methodology is worthless, or at least that what it's predicting is not very closely related to the quality of the education provided. I suspect that many Caltech undergraduates decide to pursue grad school and academia, which is not an especially lucrative career, and which is probably not highly corrolated with political leftism or "reefer madness".

I can also tell you that my friends at Rice who were looking to make a lot of money after they graduated, by and large succeeded.

In summary: Bullshit.

View on HN

>This is obviously wrong.

"The analysis goes against my gut feeling, therefore it's wrong."

Did you bother to read the article?

The Economist’s first-ever college rankings are based on a simple, if debatable, premise: the economic value of a university is equal to the gap between how much money its graduates earn, and how much they might have made had they studied elsewhere.

That's what they did, and those were the results. They even say it's debatable. Using a different analysis or metric, the results would be different.

>In summary: Bullshit.

As opposed to the anecdotes regarding your Alma Mater and rich friends.

I think he has a point, though. This study doesn't seem to control for an incredibly important variable: student interests.

If one school tends to attract students interested in making a lot of money, and another school tends to attract students interested more in academics and graduate school, can one really conclude that the former adds more value? Perhaps, but certainly not without taking into account the different student interests and inclinations towards money-making.

They do control for field of study, but I don't think this is sufficient. A school that attracts a lot of STEM majors who go to graduate school instead of industry is pretty much going to get destroyed in this study.

> Did you bother to read the article?

Yes, I did. In suggesting that their methodology is "debatable", they are indeed hedging their bets. But they are still advancing the theory that, after looking at the data, one could plausibly accept their model.

My "anecdotes" consist of four years as an undergraduate at a university (Rice) rated in the bottom 0.5%. I know damn well that Rice prepared the vast majority of students to be extraordinarily successful in the career paths they chose. I also know that my education was tremendously valuable for its own sake, and that the same was true for my classmates.

And as an academic now teaching at the University of South Carolina, I can only wish that we offered our students what was offered at Rice.

It is true, as a Rice graduate I am biased. But nevertheless I do not believe that I am stretching when assert with confidence that, according to any metric which measures anything worth measuring, Rice lies somewhere in the top 99.5% of universities in the US.

Let's be completely honest, we're all somebody's rich friend on this site.

I think the premise of the analysis is good (at least if you are optimizing for income), but the actual implementation is questionable simply because they don't have access to enough data about the students.

Rice is definitely on the way up - of any school that has a claim on being Harvard of the South they are the one that has made the most progress in terms of endowment and snagging promising researchers.

As a Caltech grad, I'll give my two cents. They are afflicted with the worse version of the same problem MIT has - "working for Harvard men". Who besides Adam d'Angelo has made it really independently big in the tech world from Caltech? They are turning out somewhat eggheadish very good Google engineers but without the reorientation towards entrepreneurship that MIT at least has with Sloan, the Media Lab, etc. Over the last 10 years, student life has become progressively more proscribed as the administration seeks to make campus a tamer version of Princeton. Caltech's purpose was to produce scientific leaders of the mold of Oppenheimer, Mark Wrighton, et al. - people who could both do research at a high level and manage institutions while providing a useful counterpart to the institutional drift that comprise NSF and NIH review panels. It's sad that the college that prides itself so much on the elite and supposedly self-selected nature of its student body and touts the supposed freedom and creativity offered doesn't trust its students. It may be that the faculty collectively take them less seriously than in the past, I don't know.

Unfortunate to hear this about Caltech.

Nevertheless, as a graduate, I hope you regard it as self-evident that, even if your alma mater has slipped, it still deserves a place in the top 98%?

I highly doubt the kind of people who would flourish at Caltech care what the Economist thinks. Which is a pity in some sense, because they should care about what the stereotypical Economist reader thinks.

Didn't they kinda address this?

"The bar is set extremely high for universities like Caltech, which are selective, close to prosperous cities and teach mainly lucrative subjects. If their students didn’t go on to extremely high-paying careers, the college would probably be doing something gravely wrong."

I dunno, how about pursuing careers as scientists rather than as hacks for the finance industry? Is that "gravely wrong"?

And as far as being close to a prosperous city -- that is because Caltech is near the world hub of the entertainment industry. A Caltech education is mostly irrelevant to finding employment there.

Isn't that an inherit flaw in the system? It is also based on the assumption that everyone prioritizes money over everything else. If I worked in the Yale marketing department I could easily spin this as a being caused by the high character of Yale graduates who are more willing to dedicate their life to lower paying but higher impact jobs like academics or public service while those greedy kids from Harvard only ever chase money.

Every other publication's college rankings prioritize "everything else" over money. The Economist just came up with a different ranking based on expected value.

It's not a problem that their ranking is based on optimizing for income, since they state upfront what they are ranking them on. However, it could be a problem that not all of the students included in the data are optimizing for that. A school that was very good at helping students increase their potential income, but that attracted mostly students who aren't interested in maximizing income would be ranked low, but it should receive a high rank.

That's not what the rankings are ranking schools by. It's entirely about value v. expected value. For example, say College State University had an expected income of $0, and the median income of their alumni was $15K. That would be a top 10 school by this methodology, despite no sane person thinking that it would represent an upgrade over going to a school with an expected income of $75K and a median income of $74K.

Think of it as a supplementary rankings system, rather than a definitive one.

The biggest problem with the methodology, to me, is it doesn't consider average cost of attendance to students at all.

Yale is also in the bottom bracket. I imagine at least Yale and Caltech have similar problems: attracting people who could be bankers (Harvard) and sending them into liberal arts (Yale) or engineering (Caltech).

Or worse (better), physics!

It's important to distinguish that this system is not rating graduates, but institution performance. Sort the table by expected earnings, and Rice is on the front page. Students admitted to Rice are smart and successful people the moment they step into classes on day one.

The question they're trying to answer is: once a smart and successful person chooses a college, how much value does that college add? Looking at that same expected earnings sorted table, the actual earnings is quite varied. MIT graduates earn 8k more than expected, and Rice earned 10k less. That's quite a difference!

There's many reasons this dataset is flawed, but not for any of the reasons you mentioned:

1. Financial aid applicants are not a representative sample of students. 2. Highly selective schools have a very high bar to exceed. 3. Students may prefer other opportunities that pay less, like civil service, or politics. 4. Earnings are only tracked for ten years (standard loan repayment).

I also question how earnings are calculated. For example, because I max out both a 403b and a 457b retirement account, my earnings are taxed as if I earned half of what I actually do. Does my double savings opportunity affect the calculation?

I definitely agree with you. I wasn't trying to explain how I think the article is flawed; rather, I was just trying to argue that its conclusions can be dismissed immediately.

In particular, I believe that your (3) describes what I see as the biggest flaw with their methodology; that (at least for elite colleges) what they are largely measuring the proportion of students that choose to go into high-paying professions.

I just find your kneejerk response of "Anything that rates my school poorly is wrong" and pocket dismissal is counterproductive to actually rating school performance. Selection bias is an actual demonstrable problem in school rankings, and coming up with some way of separating the effect of a school's selectivity from it's educational results is critical if we wish to improve the productivity of educated professionals.

I actually don't think that 3 has much explanatory power. Rather, I think the bulk of the variance is beyond 10 years. Yale's strongest suit is likely professional programs like law and medicine. These are high paying professions, but they require additional years of education beyond a 4 year bachelor's, and that subtracts earning years from the 10 year cutoff.

I don't know much about Rice, but I suspect it's particular problem is being a highly selective school in a low cost of living area. If an average Rice CS student randomly selected into other schools with comparable SAT scores, they'd be going to a much more expensive region, and their controlling for location may not sufficiently model how a CS degree differs from community college nursing.

> I just find your kneejerk response of "Anything that rates my school poorly is wrong" and pocket dismissal is counterproductive to actually rating school performance.

I see what you are saying. If their methodology ranked my school in the middle of the pack (instead of in the top 2%, which I believe it deserves), saying in effect that the advantages of a Rice education were those that its students brought with them, then I would be annoyed but I can see how that could be fair.

But the bottom 0.5%? I went there and I know better. (And as an academic I know a lot about how universities work and how they differ from each other.) I do feel that I am justified in calling bullshit immediately.

> instead of in the top 2%, which I believe it deserves

You would have to know how undergraduate education works in far more institutions than the average academic attends to have experiential data affirming that hypothesis. This is a problem in consulting academics about undergraduate college rankings, they know the researchers far better than the graduating classes, and generally rate one another that way.

> an academic I know a lot about how universities work and how they differ from each other

Which means you helped bring the average down, because grad school is 4 or 5 years of crap earnings.

As a long-time Texas resident and qualified applicant for Rice University who likely could have attended (if I hadn't refused taking SAT IIs), I'd also like to note your esteemed alma mater is likely the most picky University in all of Texas, and, if US News' numbers are correct, just a shade less picky than Johns Hopkins. In short: Selection bias is not to be ignored.

It's "ranked by alumni earnings above expectation"

It's not "ranked by alumni earning".

If you sort by Median Earnings, Cal tech shows up in the top.

we ran the scorecard’s earnings data through a multiple regression analysis

So they ran some machine learning algorithm, the Over/Under column just shows the errors in their final model. If their choice of variable was perfect, then every school would have an over/under of $0, some metrics are just not linear and not easy to get (school reputation, networking power?)

In the end this is not a very valuable metric, who cares if they think a cal tech graduate should make $82,126, it still makes $74,000. What is more interesting to me would be how much money that graduate spent on school to get there.

> who cares if they think a cal tech graduate should make $82,126, it still makes $74,000

Mostly someone who wants to measure the effectiveness of their undergraduate curriculum. As a student, there's a strong case to be made for ignoring this data and seeking the most selective alma mater possible. But if the law surrounding Duke v Griggs ever gets overturned and IQ / SAT scores are allowed in screening job applicants, you definitely want to know how much of a school's function is the actual learning and placing bit.

I wonder how Rice's presence in Houston affected their expected earnings for Rice students. I could easily see Houston really throwing off their calculations if they weren't very careful. Houston's the fourth largest city in the US, but isn't anywhere near as expensive as its peer cities. Similarly, outside of a few select industries, Houston salaries are lower than salaries in similar cities as well.

Reply on news.ycombinator.com