U.S. News pulls Columbia University from its 2022 rankings
forbes.com
forbes.com
An Investigation of Columbia's U.S. News Ranking - https://news.ycombinator.com/item?id=30603287 - March 2022 (46 comments)
As per the quote at the top of the report:
> Rankings create powerful incentives to manipulate data and distort institutional behavior for the sole or primary purpose of inflating one’s score...
It would be shocking to me if Columbia was the only institution that fudged its numbers a bit in pursuit of a higher US News ranking. Given how important that ranking has become in US culture, I wonder if it's time for the methodology to change to require audited/auditable metrics rather than continuing to allow self-reporting.
1: http://www.math.columbia.edu/~thaddeus/ranking/investigation...
I've taught at several US universities, and it's an open secret that the administrative takes steps to manipulate numbers in order to secure better rankings.
Typical example: a small, prestigious liberal arts college always has smaller class sizes in the fall semester than in the spring semester. Why? Because classes with less than 30 students are ranked higher by US News, but they only take into consideration the fall semester. No one cares about class sizes in the spring.
What a bizarre and totally unnecessary restriction.
How's it supposed to do that? Universities all already collect the data on enrollment for every class they offer.
A few years ago when 50 million went missing, I wasn’t surprised. I’d seen the budgeting process, and it was worse than you could possibly imagine.
There was an attempt to go the software route, but a nepotism deal left the budgeting department with a worse system than manual spreadsheet entry.
The pay is abysmal either way. You would make more money at an entry level software job anywhere.
In any case, the quality of the argument doesn't really matter. If the number is being gamed so trivially and yet US News continues to ignore spring classes, then the restriction has gone from bizarre and unnecessary to actively malicious. US News is basically colluding with the universities to engage in fraud.
Though shrinking the admit size is another well worn tactic, as it's the easiest way to increase how "selective" you are which is another major ranking element, plus it helps you more cheaply achieve smaller average class sizes, though of course you're forgoing some revenue to get there.
If not accepted for any of them, there's a follow-up matchmaking service (called 'clearing' in the UK) to help find an unallocated spot.
Based on a conversation with my nephew who applied to a double-digit number of colleges this past year, the application fees are apparently more the exception than the rule these days.
You can if you admit on objective metrics. If you only take people who are 215 pounds or above, you can fairly claim to reject everyone lighter than that, regardless of whether they apply.
It's a fantastically anti-consumer market they've managed to create.
If you have a given teacher, in a given classroom, with a given amount of TA help, blah blah, so that all else is equal, and then you reduce the class size, the students will do better.
The problem is that all else is far from equal in the data used in these rankings. Instructor quality drops off rapidly as you have more sections. TA help falls. The classroom may no longer be appropriate for the class, etc.
The rankings are not able to capture the effect you're talking about.
Where is this anecdote coming from? A billion years ago when I studied at a small liberal arts college with small class sizes, professors were keen to share with us exactly why they were teaching from particular texts or why they had selected certain materials to make it clear that while the class wasn't being taught by that famous guy from MIT, it was still his book and materials we were using. I knew all my professors by first name, knew where many of them lived, had access to them anytime it was needed, and that's directly counter to my partner's experience at a large university.
I went to a liberal arts college and had small class sizes (most around 9-15 in size, with the largest being 30) - I could see my professors 1:1 pretty much anytime I wanted, but there was also a preceptor (TA) for each class who didn't teach, but was available to answer questions related to assignments, projects etc. I went to the house of many of my professors for food or drinks too. Generally also was on a first name basis with professors.
The downside is that liberal arts colleges aren't research institutions and they are generally undergraduate only. This means you don't have as many choices for advanced courses. However, the professors at liberal arts colleges really tend to be there for the teaching and tend to enjoy that. One of my math professors was the head of the mathematical association of America and also authored the books on Analysis (the mathematical subject - not a data science thing) we were using.
What's ironic about this though is how many universities won't accept their own undergrads into their grad programs... IE: they know that their undergrad programs are sub-standard but they don't seem to take action to improve them because the money and the focus is on the grad programs.
There are also research consortiums of small colleges. I did original geological research 2 out of my 3 summers in a small college via the Keck Consortium:
Class size naturally dropped as you got into niche subjects, of which there were many!
Where I was asking for clarification was on the idea that somehow people were getting better materials and a better overall experience in thee 500+ people lecture hall type experiences, rather than those 20-person intimate learning style classes.
- Honors sessions - Extra seminars - Quality of Material - Quality of student support groups
All of these things get better as the class size grows. In my last class at Georgia Tech, there were 900 students studying a niche topic from a particular professor. Due to class size, there were ~12 TAs and an extremely active slack discussion group where people could get support.
A smaller class just means the professor has limited resources to assist students.
2) I guess they can actually by having massive classes in the spring and comparing to the fall.
Great Classes Implies Larger Classes (pure supply / demand); Larger Classes doesn't imply Great Classes
Controlling for as many of the relevant variables as possible, there does seem to be a strong negative correlation between class size and test scores. Beyond that, who knows, but it makes sense that small classes would succeed more often (at least up to a certain point of course)
The same is true for selectivity. It doesn't account for education at all, only the difficulty of getting in. But since it doesn't account for the applicant pool, it doesn't account for the difficulty of getting in either. You could use things like SAT scores as a measure of the difficulty of getting in and ditch the acceptance % completely. I work at an R1 that gets hit hard because we admit most applicants because, you know, we exist to educate folks, not to serve as a signalling device.
He used to say two things: that he has estimated to have taught at least 100 thousand people in his life (seems just crazy boasting, but feasible when you are working in 5-8 different schools per semester and each class has 100-250 people) and that a teacher is always teaching for the worst student in the class, and that the only reason that top universities are better is because of their selection process.
Forget class material, forget infrastructure and fancy facilities. What makes the best universities is the fact that the top students are there.
If you can not (dramatically) change your students and make them smarter, you can make your classes smaller and segment by performance. This at least stops your top 20 percentile from being dragged down by your bottom 20.
It's been my experience in college that most teachers just let the bottom quartile of the class fail (and likely switch into another major).
But in reality, even that doesn't matter. In the best universities, your worst students will be better than the average students at the second-rate institutions. If the teacher is simply giving the class to the top 75 percentile, the class will be even better still at the top uni.
How can somebody raise from the bottom 20% if they get subpar education?
Second, there can be individual mobility. A poor performing student with otherwise potential to improve might receive help to overcome whatever is bringing them down, and get them to thrive again. Even in that case, it would help them to be in a smaller class, as it's much easier to identify who among the low performers who want to be helped, and those lost causes who are a drag for everyone else.
Where this matters is in something like a lab or music practicum where the communication between instructor and students is the class. But I can tell you from experience that the variance among different years of students in a single university is large enough to disprove your uncle's point. I had one class that could have been confused for graduate students. Yet another might as well been sent by the Trump administration with the goal of destroying the institution of ear training.
The first exam that students in test-prep schools take is before classes start, usually a mock university test that can be used precisely to assess your current level and to gather information about the class overall.
You don't need to know who are the worst performers. But if you know that your class of 250 has an uneven distribution of aptitude, you'd try to break the class apart.
I've taught some college classes and guest lectured in many others. The bottom 10% is trivial to identify in a matter of minutes.
You don’t need to know which specific 50 students are in the bottom two deciles, just that they exist and that your policy is to teach to what you’ve estimated is the 3rd decile and above.
You will find out who they are in the exams, but you know they exist in next year’s class even before that class is admitted to the uni.
You don't need to know everyone's level, just a rough idea about what the worst student of the class might need.
If it’s a small group, I can actually learn everybody’s names, get to know them, personalize the material more and spend big chunks of the class just answering your questions and starting discussions based on them. The class becomes less of a mere information dump and more of a mentorship.
That was quite the course.
I went to a relatively small private engineering school. It made a world of difference in my education knowing the professors, and having them know me, especially given that with small classes the professors offered office hours where I could come and discuss the material that was covered during class, as well as problems I encountered on midterms.
I still remember nearly all of my professors, fondly, precisely because of the dynamic small class sizes allowed for.
This most often comes in the form of prolonged rhetorical questions. A teacher, fearing he is losing the class' interest might pause to ask "And, what is the benefit of the scientific method?" The point here is clearly to gauge whether the class understands and is following along in the train of thought. Usually, the rhetorical question has multiple interpretations and valid answers. And usually, the students are not so much wrong, as they simply fail to track what answer the teacher is hoping for. For example, the students might answer the rhetorical question above with "It helps us understand the world.", "It allows us to test theories.", etc. In fact, the teacher was looking for something like "It allows for the independent checking and confirmation of the findings of others."
In a real world example, this sidebar of questions might have taken minutes, and more often than not, completely breaks my train of thought. Often, the teach has completely paused a valid train of thought to either measure or enhance class participation, and in doing so has made it harder for me to understand the lesson.
Sometimes this goes even more poorly, and the teach just wants to witness a "class discussion," which can often entail spending much of the class polling the students for their thoughts. Depending on the subject, this can be a total waste of time. Most of the students in the class are poorly informed, and their thoughts are much less useful than a traditional lecture would have been.
I certainly admit that interactive methods of teaching can be particularly effective, but they are not an unalloyed good. They can often be worse than a simple lecture.
A foundation of the bloated academic and teaching industry requires that that professors can not teach more than a small number of students at once, or at least not teach them well. And that people who were taught in schools with smaller classes that were more expensive must be better. All claims about teaching and learning should be considered with that in mind, especially when it comes from the industry itself.
That being said, interactivity with professors probably works better in most other settings than a lecture format.
I actually think that putting a bad teacher in a small class makes it worse than a large one.
You wouldn't pay tens of thousands of dollars in tuition for lectures unless you were the kind of person that universities like Columbia try very hard to not admit.
Classes, however, are very valuable. You can ask questions when you don't understand things, you can talk with the other students to build shared understanding, you can get your homework graded and confirm that the things you weren't sure about on it were actually correct, you can take exams and test your skills. And those things are much, much better when there are 16 of you in the class rather than 128 or 512.
The key problem of using a measure as a metric is sometimes referred to as Goodhart’s law. The law is based on a 1975 paper on economic regulation, and is typically paraphrased as “When a measure becomes a metric, it ceases to be a good measure.” When managing a system, this problem is critical, but is deeper than it at first seems, as I’ll argue.
https://www.ribbonfarm.com/2016/06/09/goodharts-law-and-why-...
I always suspected it was a way to fudge numbers, but at the same time I knew I was genuinely on the bubble of being admitted so I took the opportunity gladly.
(My memory is that the other spring admits were, like me, from racial groups considered “over represented” so they may have been trying to affect diversity numbers too.)
The city's tracking of student attendance only lasted a portion of the year or a portion of each quarter.
During those times, much more effort went into calling the homes of missing students and finding out why they were not showing up to class.
I remember asking about why all the calls were happening or not happening and it was sort of quietly explained by the staffer: measurement of attendance affected how the school was being evaluated by the district. Presumably, budget was hooked to it in some way.
I didn't make any waves about it because "working" in this office was super powerful and the staffer always brought in cookies for us. Amazing gig.
Teachers take attendance on other days for internal use, but Count Day is special and you must not be absent!
> If instruction is cancelled on count day due to conditions not within the control of school authorities, with the approval of the State Superintendent, the affected instructional programs must use the immediately following day on which the district resumes session for count purposes
per the rules at:
https://www.michigan.gov/-/media/Project/Websites/mde/Year/2...
I do recall my cousins at a poor rural school in Barry county having a couple snow days on Count Wednesday, and then subsequently calling one on Thursday when it wasn't really required and another on Friday where it really wasn't required, so that counts were as high as possible on Monday. Their snow days were much more frequent due to bad roads and bad road maintenance, and their state funding was much more important compared to my wealthy district.
It was an open secret when I was in Boston that Harvard actively encourages people to "give it a shot" when they know full well they're not getting in to boost that metric.
1: https://www.nytimes.com/2019/11/29/us/harvard-admissions-rec...
Though, maybe it's just confirmation bias and ego. Maybe there was something in my application that gave Michigan pause but seemed fine to MIT. At that time, Michigan did ask which other schools to which you were applying. I'm not sure if that's still the case.
It's still frustrating as both his parents are alumni. Thanks to Covid and test waivers, 2021 seemed like a rougher year than normal to be applying to the higher end of schools.
It's basically saying "other people seem to think this college is good, so we'll nudge them up a few points"
In other words, it's attempting to measure public perception about the college instead of the actual results it produces. How the public _thinks_ a certain college is shouldn't factor into how good you _tell_ the public that that college performs
Assuming that the college is selecting students for attributes that you are looking for in fellow students, then the college that denies more people will have students with more of the attributes you are wanting to be around. So it isn't just about perception. There is some math behind it as well.
Except "average SAT score of incoming class" is another field of the rankings. All the measurable criteria are handled as independent variables - there's no need for a poor proxy.
So does page rank or citation index - popularity begets visibility, begets popularity.
I mean it’s a pretty darn conservative proposition to say “we think that averaged over the entire population of applicants that the schools that students spend their time applying to will trend, however small, to better schools.
Where the flip side of “it’s all noise and it’s impossible to evaluate a school better than flipping a coin without actually attending” is an incredibly strong assertion.
It's like the XKCD about Wikipedia entries being cited in articles, which get cited in papers, which then get cited in the same Wikipedia article.
The rest is bunk. Undergraduates aren’t connecting with famous professors. A faculty team with deep expertise in a particular area is relevant but not in the rankings.
It doesn't do a great job comparing across categories, but then, neither does US News...
Maybe they'd be richer if their admissions was more open about taking quasi-bribes, but I suspect you're dramatically underestimating the orders of magnitude of difference that appear on this metric. You find clear tier gaps.
Don't get me wrong, this is a silly proxy metric about three degrees out from anything you'd actually want to know. But it gets surprisingly good results for how simple it is to tabulate.
If you have the big name degree it might push me to interview you when otherwise I'm borderline about tossing your resume, but that is it. I need to know you are good not your school.
The common app has made this worse. In "the old days" you had to fill out a full application, write different essays etc.
My anecdote is that I was once recruited by one of the best departments in the world in my field, at a very junior level ... and then, before I started, my contract was rewritten to be at a higher level, because they'd found out that that would allow them to count my grants in their metrics. Not only is this obviously dodgy, but I think it's actually illegal in Australia (since better-qualified candidates weren't given the opportunity to apply for the post at the higher level), but they did it anyway.
Ripe for abuse...
If statistics were distributed in open format, transparently, and there were multiple rankers (each with their own weights and mixes), universities would be less incentivized to game "the US News metrics".
We already have what you suggested as a solution, and look at where we are. The problem is more fundamental.
[1] https://collegescorecard.ed.gov/search/?page=0&sort=threshol...
The fine print:
>The median annual earnings of individuals that received federal student aid and began college at this institution 10 years ago, regardless of their completion status.
So for the anxious parent, not a lot to go on.
https://en.m.wikipedia.org/wiki/Doping_at_the_Tour_de_France
The most damaging thing you learned in school wasn't something you learned in any specific class. It was learning to get good grades.
That's the problem with the quote. Grades may not be the most important thing but they can, and often are, the most important thing for any given individual. For every Bill Gates that dropped out, there are a million John Doe's drowning in student loan debt with no degree and limited future prospects to show for it.
How is the thing you say a problem with the quote? These can both simultaneously be the case, and often are:
1. The most damaging thing John Doe learned in school was learning to get good grades.
2. Richard Roe didn't get good grades and is drowning in student loan debt with no degree and limited future prospects to show for it.
You have offered no reason to believe that if (2) is more true, or true in more cases, (1) is less true, or true in less cases, or less important. In fact, this tighter conjunction can also often be true:
1'. The most damaging thing John Doe learned in school was learning to get good grades.
2'. John Doe didn't get good grades (despite being damaged by learning how, he didn't learn how enough, or other things interfered) and is drowning in student loan debt with no degree and limited future prospects to show for it.
You've also offered no reason to believe that if (2') is more true, (1') is less true or less important.
I have an idea! Maybe you're misinterpreting the quote as meaning that, if you're in school, it would be a good strategy to stay there but not learn to get good grades. In that case there would be a sort of conflict: by diminishing the damage from (1') you may be increasing the risk of (2').
But that isn't what the essay advises at all, and I'm puzzled as to how you could have read it as suggesting that. It explicitly advises students to do exactly the opposite; it advises them to get good grades by hacking bad tests in the usual ways (cramming before exams, etc.) because everyone will judge them by their grades.
But that's not really the point of the essay, and it would be a pretty boring essay if it was. The point of the essay is its advice to people who aren't enrolled as students: unlearn how to get good grades, and instead of focusing on how to make your product look good, actually make a good product. Also, and maybe more importantly, try to work in fields of endeavor where the way to win is to do good work, rather than hacking bad tests like you do in school. (This might suggest not going into school at all, which would at least avoid the student loan debt problem.)
"Personally, I liked the university. They gave us money and facilities, we didn't have to produce anything! You've never been out of college! You don't know what it's like out there! I've WORKED in the private sector. They expect results."
Going to college optimizing for "learning" and to gyms for "fitness" are poor use of time and resources.
The essay is just pointing out that strategy to hack tests for grades stops working. You need to improv on it in startup land or in the kingdoms of BigCo.
[1] https://www.parchment.com/c/college/college-rankings.php?pag... [2] https://www.topuniversities.com/university-rankings/world-un...
According to their methodology[1] page, it looks like for student who gets multiple offers they do an Elo style ranking. The first problem; and which probably explains the weight to military academies, is that students tend to self select into tiers of colleges, so a group of students who are interested in the Air Force might not even apply to Harvard (even though the USAFA is a competitive school) and upon admission opt to go there (after all if you even were even considering of joining the air force, why would you reject your one shot there to go any other school)?
[1] https://www.parchment.com/c/college/college-rankings.php
As a result, if you have your sights set on anything other than the two in-state schools or BYU (strong Mormon influence here) the counselors don't know what to do with you and you are on your own.
It worked out and I have no doubt my kid will have strong opportunities post-college but he missed out on any chance at the truly life-changing schools, which I have no doubt he would have succeeded at had he gotten in.
LOL wut? Stanford is modestly well known, but obviously Harvard is the one that is unambiguously first.
Is that no longer Yale? From the perspective of someone who had a rather poor exposure to universities during my high school years—so, precisely what someone gets from pop-culture and osmosis without anyone in particular acting as a guide—my impression was that Harvard and Yale were unambiguously the Big Two, and everything else fell somewhere after them in terms of recognition and prestige.
Is this a CS perspective? I hear a lot about Stanford's CS program so I assume it's notably good.
A lot of the elites are hype-over-substance, and it's often hard for a naive audience to screen out which ones.
I have significant experience with HYP, less-so with Stanford. I don't think I am a naive audience, I know many people from these schools and have attended classes, either due to being a student there or online videos in the case of Stanford & Princeton.
I'd never have gone to Yale because it was a poor fit for my interests. That's different from being a poor school. Overall school quality is less important than fit. Given my interests, I would have pick Stanford over Yale (and in turn pick Caltech, CMU, or Georgia Tech over Stanford).
That said, Yale is a phenomenal experience. It's very academic in the classic sense -- deep, philosophical discussions, and thoughtfully exploring topics in a way that's increasingly rare in the fast-paced elites. It gives a lot of what people assume they'll get out of a small liberal arts college (which is different from the reality; most small liberal arts colleges are expensive and horrible).
I have little knowledge or Princeton. I know both Harvard and Stanford in great depth.
Stanford tied with Harvard for 6 Senators in the 117th Congress, impressively, but obviously Harvard dominates that metric over a longer time.
Harvard still has more than twice as many billionaire alumni notwithstanding how many billionaires these days are in the tech industry.
Also, this is a thread that's largely about undergrad, so I'm not sure Rayiner was talking about law school.
There are different rankings for law schools, and Yale has been considered the most prestigious for many years. I think SLS nabbed the top spot various times in the last decade, but Harvard is definitely not the uncontested leader in law schools.
Cambridge has a reputation for strongly preferring UK students.
> Almost any numerical standard, no matter how closely related to academic merit, becomes a malignant force as soon as universities know that it is the standard. A proxy for merit, rather than merit itself, becomes the goal.
https://austingwalters.com/foia-requesting-100-universities/
Often I’d find the person in charge on linkedin and email them directly (work email obviously). And dig around until I found the particular person.
UIUC put up a particular fight when I FOIA requested information about their post-graduate surveys. They’re definitely manipulating there. I was in the process of filing a lawsuit when covid hit and gave up because I wasn’t sure if I’d need the funds (plus court shutdown at the time).
FOIA only applies to federal government agencies, universities (other than ones operates by the federal government, like the service academies) aren't subject to it.
State universities might be covered by state sunshine laws that are similar to FOIA, and generally the laws and policies around those only require a request from the public and aren't too particular about form or citing a completely wrong and irrelevant legal basis, so in that case an “FOIA” request might actually have some legal effect. But most private universities would just see it as a particularly pompously written discretionary request.
But I am really curious about the argument against the requested survey data. The poster linked to his web page which specifically lists UIUC as being exemplar and providing data without even the need for a formal FOIA request.
I’m sure they knew that lol
I requested: country of origin, gender, degree, gpa, salary range, field of work and a couple other fields I forget now.
> No one should try to reform or rehabilitate the ranking. It is irredeemable. In Colin Diver’s memorable formulation, “Trying to rank institutions of higher education is a little like trying to rank religions or philosophies. The entire enterprise is flawed, not only in detail but also in conception.”
A lot of HN commenters here trying to "fix" the details of the rankings.
Here’s a classic case where if you do something long enough, people will trust you.
I don't know what the equivalent today would be. The Reuters website maybe?
https://www.bostonmagazine.com/news/2014/08/26/how-northeast...
Did they graduate in higher numbers? Did they have better career outcomes? Did more get into better(?) grad schools? What is their median income? Are they happier, did they enjoy college more?
Northwestern was already relatively prestigious.
As for what the improvements are like... hard to say. Some ways to improve the rankings like getting more students you don't want to come to apply (so you can appear more selective) don't actually help at all, while things like small classes even if done to "game" the rankings could actually help.
There's also just the cyclical aspect of things: the quality of your experience is going to be significantly determined by the quality of the student body, so if you have a better applicant pool because of your better rankings you maybe have a bit of a virtuous cycle that's somewhat disconnected from whether the things you changed actually improved anything real on their own other than the ranking.
In my opinion as a student who entered Northeastern in the mid 2010's the answer to all your questions is an unequivocal yes. More competitive student body, better profile to attract employers, more student amenities to enhance on campus life (we could have more in this regard imho). Compared to the commuter school of yesteryear it's for sure a much better educational experience.
If a school only admits students who have at least two years at another college with high grades, their graduation numbers will skyrocket.
A highly selective college, like Harvard, could even refuse to grant credit for those two years.
There are less nefarious ways of selecting a class of likely graduators: leadership in extracurriculars is highly correlated with post-secondary graduation.
Foreign students with limited English fluency have very low graduation rates. They could (should?) be excluded.
Those are officer training schools, so it makes sense they're ostensibly selecting for leadership skills, because they'll be overseeing enlisted recruits.
And make sure you control for your inputs. It's harder for the #2 campus of state school to wring "good outcomes" from the children of plumbers and truck drivers than it is for big brand name schools that have plenty of kids of doctors and lawyers in there to drag their average up because even if you do a middling job educating them and have squat for industry connections they will still land internships and go on to get good jobs.
the university definitely has done things to improve its rankings over the past ~30 years, some legitimately good (hiring higher quality staff, putting more emphasis on research, building more student accommodations like on campus housing, expanding the co-op program) and also less than savory moves (the NUIn program which admits more freshman students without having to record their statistics in the incoming class record, international campuses to "build the brand"). I think, at least the CS and co-op programs are great, and - minus the cost - is a overall great school. they definitely had the aim of improving the ranking, but in doing that they did improve the quality and size of the school along many important fronts. my read is then that USNews still has use in measuring universities, up to a certain point (say top 50 or top 100)
The changes NEU made between 1996 and 2006 were seemingly genuinely good for students. My dad had graduated from NEU in '86, and he was really impressed with how far the school had come. But I wouldn't say anything that changed while I was there had any real impact on my experience, so my gut instinct says they were more gamey than anything.
No one should try to reform or rehabilitate the ranking. It is irredeemable. In Colin Diver’s memorable formulation, “Trying to rank institutions of higher education is a little like trying to rank religions or philosophies. The entire enterprise is flawed, not only in detail but also in conception.”
Students are poorly served by rankings. To be sure, they need information when applying to colleges, but rankings provide the wrong information. As many critics have observed, every student has distinctive needs, and what universities offer is far too complex to be projected to a single parameter. These observations may partly reflect the view that the goal of education should be self-discovery and self-fashioning as much as vocational training. Even those who dismiss this view as airy and impractical, however, must acknowledge that any ranking is a composite of factors, not all of which pertain to everyone. A prospective engineering student who chooses the 46th-ranked school over the 47th, for example, would be making a mistake if the advantage of the 46th school is its smaller average class sizes. For small average class sizes are typically the result of offering more upper-level courses in the arts and humanities, which our engineering student likely will not take at all.
[1]: http://www.math.columbia.edu/~thaddeus/ranking/investigation... (section 8)
https://jeffhuang.com/computer-science-open-data/#bias-in-co...
For instance SIGCHI 2021 had 28 best papers out of 747 accepted papers (or 3.7%) whereas CVPR 2021 had one best paper out of 1660 accepted papers (0.06%).
I have no opinion about whether it's "better" to be stingy or generous with best paper awards. But obviously any kind of ranking that doesn't account for differences between conferences and subfields is going to be quite suspect.
To put another way, to count in a way such that a CVPR best paper is worth 28-fold a CHI best paper is also quite suspect. There's a rabbit hole you can go down to find the best conference to submit to where the submission-to-best-papers is low, to optimize for this.
PS: note that there's an upper bound to best paper awards, which is "< 1% of submitted papers".
Just because you can't get an exact measurement does not mean that a metric does not exist or is not useful.
The top 10-15 offer an almost indisputable advantage with the top 5 or so being a tier unto itself. Anything outside of those groups is largely "it depends" and 50-100 forms another tier where total cost of attendance largely dictates whether one school is "better" than another.
Did anyone try to make a ranking for religions similar to those for universities? Sounds like it would be a fun project making a great point.
Measures like "Fees", "Diversity", "Alumni Salary", "Financial Aid Provided" would all be interesting to see for each religion.
Universities are mostly unimportant in terms of what they offer: class sizes, education, curriculum, etc... All mostly bs that doesn't matter.
The real role of universities is to gather together smart people as they develop. This requires mostly a sort of self-selection of applicants, who need to agree independently to go to the same university. Hence rankings, prestige, and all that nonsense.
Readers of US News would more likely start to lose trust in the rankings rather than move their assessments of the Ivies downward, right?
[1] https://arstechnica.com/information-technology/2014/09/why-t...
If none of the Ivy’s are on US News, well, it’s probably a shitty ranking.
But if e.g. just Penn falls off the list… most people are just gonna assume that Penn got worse.
Not trying to be contrarian, but I had two friends literally pick between Ivies based on this very list (in 2015).
It’s generally accepted as “The List” by a lot of people. If you’re a parent with no other frame of reference, The List has a serious impact.
In any case, I bet Columbia will keep getting more really high quality applicants than they have seats.
If you got into #4 and #40, then yes, you were going with #4.
If you’re of a certain background, it’s really your mom or dad’s alma mater, and then one of a few “perfectly acceptable, fine schools.”
I think the difference between say #4 and #12 in a given year exists in the minds of middle class strivers. I went to a public university and a private one for undergraduate and graduate school, and at the top levels it really comes down to the professors on an individual level and perhaps the department, more than the institution itself.
And in any case, if you pay attention to the finer movements in the rankings, schools rarely keep the same position between publications. It’s just the same ones playing musical chairs with each other year after year.
Honestly though, college rankings are toxic, as is this whole prestige economy we've built around them. I don't even really care about colleges faking numbers, so much as other shady practices like sending students false promises to get their hopes up - pretty much for the express purpose of rejecting them down the line. As any unfortunate student who has either had to apply to college, or been the parent of someone applying, you can't avoid the "chance me" threads or the endless HYPSM dick measuring contest while trying to get legitimate information.
[^1]: https://www.nytimes.com/2022/06/30/us/columbia-us-news-ranki...
Also, US News and World Report rankings have major problems, for instance, the ranking algorithm considers campus aesthetics and food quality, but doesn't account for price, giving schools incentive to raise prices to fund campus improvements that boost rank, in turn boosting applications, in turn reducing % acceptance (the principal indicator of "quality" in US schools). This was how TCU went from a meh christian school to a "selective private college" in just a couple of years. World Report now publishes a "best dollar value" report to account for this, but few read it.
wtf
You can see a lot of hypocrisy that’s uncritically accepted by a lot of organizations, including nonprofits. Exclusionary higher education is a particular notable example, given the soaring rhetoric of “inclusion” spouted by some people involved with higher ed, versus the reality of those same schools seeking to reject as many applicants as possible. Princeton University’s president, Chris Eisgruber, has, for example, blathered extensively about the school’s efforts to “combat systemic racism.” Princeton has a $37 billion endowment. The school’s undergrad acceptance rate is 5.6% and it charges a sticker price of $73,000 a year (yes, the school does accept a handful of token low-income students every year, but that the school’s overall demographics reflect its target: the wealthy). Does that sound like a school devoted to combating systemic racism to you? How can people make these kinds of arguments with a straight face? Colleges and universities are run largely for the benefit of their administrators. The other exclusionary schools are doing the same things, as are their private-school feeders, despite their vigorous marketing to the contrary.
Regarding the above paragraph, let me be clear: describing how something is, is not the same thing as approving of it.
Also, even the value rankings are mostly just Ivies at the top. Funny enough, my school is in the ~30s for overall rankings, but in the low 100s for value, since they're looking at the out-of-state cost (which for UCs is around 50k vs. 15k in-state), and they include grants, which is extremely variable by individual student. If they looked at the in-state tuition, UCs would dominate the value rankings.
https://www.usnews.com/best-colleges/rankings/national-unive...
https://www.usnews.com/education/best-colleges/articles/rank...
The ranking formula is also changed arbitrarily based on the reactions of colleges and readers. My school went from #9 to #1 one year, there were a lot of "wth??" reactions, and the following year the formula was tweaked so that our ranking dipped down to #4. I'm pretty sure nothing changed materially in the school over those 3 years, but the ranking moved around a lot.
The whole system is a farce.
* I haven't researched freshman vs transfer data myself, but it's something I've heard from multiple startup founders in the college education space.
https://www.bostonmagazine.com/news/2014/08/26/how-northeast...
There are probably some significant parallels to any executive who manages to game the compensation structure at any large corporation to their financial benefit.
I’d have to find it but someone did an analysis of folks who got into elite schools and didn’t go for whatever reason and saw that by mid career they had more or less the same outcomes.
Thiel Fellows entire premise is around this effect, as well.
An Ivy League school does not need help from the rankings -- but rankings that leave out one of the most prestigious and well-known universities in the country are useless.
I tend to agree, but why then did Columbia fudge the numbers?
I think there’s some envy from the “lesser ivys” that they wish they were Harvard or Princeton. A high USNews ranking proves little, but I guess Columbia wanted to be tied with Harvard.
Once everyone starts optimizing for a metric, that metric ceases to be useful. It's only a matter of how long it takes people to realize that this has happened.
Unfortunately, yes
> superior
Key word unfortunately
I think this is kind of what's happening with Columbia here. For example, Columbia claimed that 100% of the faculty had a terminal degree. Prof Thaddeus (if you read his full blog post) questioned this from two angles. The first was to say it can't possibly be true. The second (which I found more interesting) was: why would you even want this to be 100%? Seems like Columbia has valuable faculty who don't have a PhD or MFA or whatever is the "terminal" degree int he field.
The harm here isn't in wanting a terminal degree for a high percentage of faculty, it's trying to get this to 100%. I'd say that it probably is a good sign, overall, that most of the faculty has a doctoral degree, as this is the main training degree for research and teaching. But the way US news does it, you get more points for 95% than 90%, more points for 99% than 95%. Seems like a lot of harm comes from trying to wring out that last bit, since even if you do generally agree with the value of a PhD for research (or other terminal degree), it does seem like you'd build a stronger faculty overall with the ability to hire 1 in 10 or 1 in 20 based on some other credential.
So much else fits this pattern. Small class sizes.. kind of the same thing. Sure, it's a good sign, but they're only an indicator, not a goal in and of themselves. Low admissions rates, high test scores. All good metrics, but things that can become not just harmful but harmful and maybe even fatal to a university if pursued with a single minded intensity as if total purity in the metric is the goal.
Honestly, overall we just have to reject the US news rankings. I appreciate what Prof Thaddeus did here, and it was a useful and very well reasoned and sourced takedown. But now and then, I realized that there isn't much to be added, just new and interesting ways to say what everyone knows at this point - yousnoozeandworlddistort rankings are pretty well idiotic. I actually think college rankings in general can be useful if managed cautiously and read critically, but this one is just a turkey.
But like, people don't find out about Columbia from this list, right? It is, uh, older than the country "U.S. News" is named after and an Ivy League school. Not having them on the list just reflects poorly on the list.
I dunno, nothing about advertising makes sense to me, so I guess I have trouble evaluating this kind of thing.
Common sense would tell you that you don't give 4 billion dollars to the guy who says "I'm really good at convincing people to part with their money -- whether they should or not!" But then I don't run Coca Cola, so I guess they've figured out something I've missed.
MIT vs Harvard vs Stanford are all amazing programs but are better for different types of people/interests/goals
They don't have good sports teams, no way a person in middle america would have heard of them if not for school rankings
* No I have no connection to the school except I do have some friends who teach there and some others who attended decades ago.
That said I agree that being an Ivy in NYC certainly was a major contributor to its more recent rise in the rankings and applications.
And if US news were serious they could simply have kicked columbia out for a year, but it would have hurt them to do so. They'd probably do it for a small college ranked 100 or below though.
I'd be embarrassed for Columbia that they gamed the numbers. Outside that small n though, I presume gaming the numbers is commonplace.
https://www.pushkin.fm/podcasts/revisionist-history/lord-of-...
- Small liberal arts school
- Is ranked Division 3 in the NCAA
- Realized they are losing students
- They start a football team!
- Why? For a layout of $500,000/year for a coach + staff, using existing facilities (e.g. soccer stadium) etc they get ~100 additional students who want to be able to say "yeah, I played football in college"
100 students * full boat tuition is nothing to sneeze at and is a pretty good deal from an ROI perspective. It's not a small leap from "let's drop some cash for a very obvious improvement to our attendance" to "let's fudge some numbers that no one may ever really look at" for the same outcome.
The former students lead privileged lives: they are very selectively chosen, boast top-notch test scores, tend to hail from the wealthier ranks of society, receive ample financial aid, and turn out very successfully as measured by graduation rates.
The latter students are significantly worse off: they are less selectively chosen, typically have lower test scores (one surmises, although acceptance rates and average test scores for the Combined Plan and General Studies are well-kept secrets),43 tend to come from less prosperous backgrounds (as their higher rate of Pell grants shows), receive much stingier financial aid, and have considerably more difficulty graduating.
No one would design a university this way, but it has been the status quo at Columbia for years. The situation is tolerated only because it is not widely understood. "
https://www.math.columbia.edu/~thaddeus/ranking/investigatio...
The problem, of course, it that the timing doesn't make sense at all. You say this happened in the early 50's. That can't be the early 1850's, since Bader wasn't born yet. But by the 1950s, Feynman was already long gone from high school. Indeed, he'd already devised path integrals, and so he certainly knew about the principle of least action!
Perhaps you meant 30's?
The only criterion that matters for a student that wants to select a university is the fraction of graduates that go on to do impressive things.
Adam Ruins Everything, Season 2, Episode 7 [1] 1: https://youtu.be/EtQyO93DO-Q
I guess there's still hope if higher education can learn from Professors of the likes of Michael Thaddeus. He and others like him make pursing a degree valuable. The administration can go...
Note: read the original document: http://www.math.columbia.edu/~thaddeus/ranking/investigation...
This whole entertainifying business really isn't good for anyone, except perhaps for people making money off of numbers, which is still not good for most.
A FTC review of college advertising would be interesting, if they are selling you a degree with basically no career prospects while proudly displaying these gamed rankings, that sounds like fraud and misrepresentation. Perhaps the students can get a refund?
What about a list of independent|objective tests(for the 100 different majors out there) to rank students from any schools when you graduate that acts like SAT|ACT for jobs? do make it optional though, if employer finds that valuable, more college graduates will take it, it will work better than leetcode I hope.
from TFA: "on Thursday of last week, just before the July 1 deadline for submitting new data to U.S. News... Provost Mary Boyce, said [Columbia] would not submit data this year, adding that the university had “embarked on a review of our data collection and submissions process”
and that is why they will not be in the rankings. It seems Forbes filled out this submission just like Columbia did theirs, to skew overall appearance.
the "scandal" could very well still be the scandal everybody thinks it is, but it seems to be an overstatement to draw the bold lines that Forbes is juicing their article with. And as others have said, unless we know the extent of fudging at other schools it's difficult to judge the severity of this case.
full disclosure, I have no Columbia or ivy affiliation, but I perceive it to be easily within the top 10