Going to elite Indian colleges improves earnings, but not test scores | Hacker News Reader

Going to elite Indian colleges improves earnings, but not test scores

economist.com

141 pointsblueblisters91 comments

Important caveat: The test in question seems to be the exit score of junior colleges, equivalent to 12th grade education.

Which suggests that the discontinuity found might very well be due to the educational quality of the "regular" college. It would, likely, only get rid of general intelligence as confounder. This, however, would just make educational quality more pertinent rather than less.

Edit: Well I found a working paper from 2011. The matter is more complex.

Apparently, in India, both public and private colleges exist under the heading of a single local university. The students at private and the more prestigious public colleges then take the same exams in Bachelor.

The authors have collected data from one university and a couple of colleges. For that sample, they control for ability with the entrance scores, but then match the regression discontinuity on exit scores.

That is: Public colleges have on average higher exit scores. The paper shows that this is due to having stronger students (entry exams). Controlling for this, they find no difference in the "educational trajectory" of public vs. private colleges, however, public colleges then have better job outcomes.

So my earlier caveat should read: The study shows results for one Indian university (with several colleges) in the field of liberal arts.

Whether the study is externally valid for IITs I'll leave up to the reader.

Get it from https://sci-hub.tw/10.1257/app.20140105 :)

I went to an "elite" Indian college. Anecdotally, I can say the quality of education was quite a bit higher than what could be had at non-elite colleges. I can also say with reasonable certainty that the large salary difference is mostly due to signalling. Even the graduates from a top college has much to learn before they can contribute to their job.

Also, elite college admissions are based on test score on one day in the student's life in high school, while their brain is still developing. I have personally seen many who didn't qualify, but have gone on to do great things. Their initial salary was definitely low.

Completely agree with your assessment, I also went to an ‘elite’ college, had the exact same experience.

To add to this with numbers the US military imposes a standardized test on all service members and many high school students who never become service members. The test, the ASVAB, is composed of several sections but all anybody really looks at is the GT score.

https://en.wikipedia.org/wiki/Armed_Services_Vocational_Apti...

The first time I took the test as a dumb 17 year old who was at risk of failing out of high school my GT score was only 107 (out of 130) and my average performance placement compared to all other test participants that year was 79 (on a scale of 0 - 99). Generally, everybody who entered the military as a teenager notices dramatic improvement on that test when retaking it later life even with no preparation.

When I wanted to become a warrant officer I had to retake the test because of my low scores. The minimum GT score to become a warrant officer is 110. I retook the test at the age of 35 after 8 years of being a full time software developer in the corporate world. My new GT score was 129 (out of 130) and my placement amongst other test takers that year (on a scale of 0 - 99) was 98.

I think research generally bears this out across a variety of metrics and meaures including violent crime data. The human brain is still developing until around age 24 give or take two years. This is why applaud California's recent decision to halt standardized tests, such as SAT and ACT, as a mandatory placement criteria for entering university. All those tests measure, given the youth and maturity of traditional high school students, is extreme preparation and not performance or potential. Preparation in that regard is a socio-economic stratifier as it takes time and money outside of the student's own initiative and state provided education.

> This is why applaud California's recent decision to halt standardized tests, such as SAT and ACT, as a mandatory placement criteria for entering university. All those tests measure, given the youth and maturity of traditional high school students, is extreme preparation and not performance or potential.

Unless you think that universities should pick their students by lotteries, you must answer how whatever that replaces SAT is not worse. Because whatever replaces it, for example identifying well rounded candidates with extra curricular achievements is even more socio-economic stratifying.

It largely feels like an excuse to lower the competitive bar to manage the demographics, allowing both more legacy hires and more representative demographics. Such actions however will come at the cost of merit.

> is even more socio-economic stratifying.

Is there data to suggest that? Public schools are generally very good about providing a variety of extra-curricular activities to keep students focused upon athletic and scholarly activities outside the classroom if the student is willing to participate. Contrast those tax funded and faculty lead activities to private tutors and rigorous SAT practice that is almost exclusively limited to families with the personal financies to afford such.

> It largely feels like an excuse to lower the competitive bar to manage the demographics

I can certainly empathize with that. Wealthy households had an advantage with private tutors and timely dedicated/forced test preparation activities. Now that advantage has evaporated so that wealthy students are now less exclusively advantaged, or in other words the bar of entry has lowered to wider availability.

> Public schools are generally very good about providing a variety of extra-curricular activities to keep students focused upon athletic and scholarly activities outside the classroom if the student is willing to participate.

I think you’re greatly underestimating how much the college admissions game has changed in recent years. For top universities (like the better UCs, Ivies, etc.), being, for example, president of a club in one’s high school is essentially irrelevant. The paths to admission consist of: (1) being a top athlete, (2) significant extracurricular achievement (meaning state/national accolades, founding a non-profit, etc.), or (3) extreme adversity.

You’re absolutely correct that the SAT rewards those with wealth and resources the most. But the absurd time and resource investment required of modern high-schoolers and their families to participate in extracurriculars in the way top colleges now demand is far greater a burden than SAT prep.

Lottery with a min bar would fix most of these issues, especially the crazy extra curricular game that is played today.

Most of the Ivy League schools already implement a lottery, because even with the absurd requirements for entry they can't differentiate the qualified candidates well enough to make a determination given the limited number of acceptance slots and high number of applicants.

You have to keep mind those schools are extremely tiny compared to the highly rated state schools in California, Texas, and Florida.

This suggests that issue is not so much who get to Ivy League school, but the disproportionate advantage Ivy League schools graduates have. No matter how the entrance test is tweaked, as long as not studying there is a block in some career goal of an highly talented and highly ambitious student, there will be issues.

> You’re absolutely correct that the SAT rewards those with wealth and resources the most.

What, why? Not American here, but in the past I was looking into going for university in US and studying for SATs and it's just a test that everyone can learn for...

Lower socioeconomic status is thought to add stress, often in the form of less time for studying because teenagers at the lower end of the socioeconomic ladder have to work more. They also can't afford tutors, have fewer books in the house, etc.

Allegedly, with time, money, and tutors you can "study to the test" pretty effectively.

Yes, everyone can study for it, but generally the easiest ways to boost your SAT scores are to pay a tutor to drill you through the parts that can have the biggest impact on your scores, and to take the test multiple times until you get a score you like, both of which can get very expensive. Retaking the test is really helpful, as you'll be more familiar with the format and less stressed out the second time around, but a lot of students don't have the opportunity to take it more than once.

> It largely feels like an excuse to lower the competitive bar to manage the demographics, allowing both more legacy hires and more representative demographics.

It doesn’t seem like a good solution to that problem. How will removing test scores as a factor help smart kids of underrepresented minority groups against dumb rich kids (with influential parents) who are too lazy to study for the SAT? It seems much better to keep the test but apply a corrective factor if you believe it’s biased against some group. It’s the only way I can think of to keep the dumb rich kids out.

The point was that they don't want to keep dumb rich kids out. The parents are an important money source and you don't want to upset the ruling elite by withholding from them the prestige they are used to.

Public schools aren't for the "ruling elite"

The point is to replace this objective transparent measure with a subjective opaque measure. Like alumni interviews, and ranking based on personality. That way they can twist the results a preferred way

> Unless you think that universities should pick their students by lotteries

This is already the case in the US - more so than any other developed nation I think, because of how poor state public schools can be and the lack of a welfare / healthcare safety net for the poor. The lottery of who you are born to and where you live is pretty much the main determination of whether you get into an elite university or not.

The answer is to provide enough high quality higher education so everyone who wants to can get one, as long as they meet the minimum bar (pass GRE / high school diploma). Let people prove themselves as adults. I'm convinced this can be done, but probably not in the traditional way of requiring a physical presence on campus.

I don't trust anyone who thinks a "competitive bar" to education is a good thing.

There is a competitive bar to getting an MIT education. And that is a good thing, because it encourages competition and excellence.

It is not a competitive bar to education, but a difference of great and not so great universities, which are going to choose students from great and not so great candidates.

> Unless you think that universities should pick their students by lotteries,

The British Open University (distance learning) has no entrance requirements and still has a good reputation and their graduates are successful in getting well paid jobs.

It would be quite reasonable to use a lottery if the number of applicants exceeds the number of places, those who discover that they are not good enough or are not willing to work hard enough will drop out quite quickly. You just need to dimension the first year teaching capacity a bit higher than subsequent years so that you have enough students left at the end of the first year to fill the subsequent years.

I'll consider your point valid if oxford and cambridge also did the lottery. Otherwise, if the elite universities, which are likely to have a lot of good candidates, won't do a lottery, maybe there is some merit in choosing people by merit.

> This is why applaud California's recent decision to halt standardized tests, such as SAT and ACT, as a mandatory placement criteria for entering university.

https://www.cnn.com/2020/05/22/us/uc-suspends-sat-act-for-ad...

It's not at all obvious that was done, and there are a ton of caveats in the new rules. They basically want a shot at developing a different test. They didn't eliminate testing.

"We are removing the ACT/SAT requirement for California students and developing a new test that more closely aligns with what we expect incoming students to know to demonstrate their preparedness for UC," Napolitano said.

"If a new test does not meet the specified criteria in time for admissions for fall of 2025, UC will eliminate the standardized testing requirement for California students, according to the news release. UC will work on a separate approach for out-of-state and international students."

Why speculate when people have already done research on this.

https://www.wsj.com/articles/the-truth-about-the-sat-and-act...

> Myths abound about standardized tests, but the research is clear: They provide an invaluable measure of how students are likely to perform in college and beyond

>All those tests measure, given the youth and maturity of traditional high school students, is extreme preparation and not performance or potential.

The tests in essence measure either intelligence or work ethic. Work ethic is probably more important for college performance than anything else to be honest however you can do fairly well on the tests with intelligence alone. The point of college admission isn't really to predict your performance in life but rather to predict your performance in college. If you become a better person at 24 or 35 it doesn't really matter since you're long gone from college by then.

Work ethic isn’t just working hard. Slaves work hard without work ethic. Work ethic requires self initiative. There is no reason to suggest that those high school students who practice hardest for the standardized tests do so entirely of their own accord with a forcing function from adult interference.

Nothing is a perfect proxy for something else so lacking an oracle or a time machine all you can do is try to find an approximation. Also, adult interference doesn't end just because someone goes off to college.

But that also applies for grades as well. High school is preparation for college. Shouldn't colleges select for people who are prepared over those who are not? Should we get rid of grades, extracurriculars, etc? What should colleges use then? Just legacy admissions?

> I think research generally bears this out across a variety of metrics and meaures including violent crime data. The human brain is still developing until around age 24 give or take two years.

So then let those people reapply for college when they "mature". What does that have to do with SATs?

> Preparation in that regard is a socio-economic stratifier as it takes time and money outside of the student's own initiative and state provided education.

This is hilarious. SATs and ACTs acted as an objective factor against "socio-economic" factors. High school grades are unreliable as different schools grade differently. It also prevents against wealthier students buying better grades by paying for people to do their homework for them. It guards against wealthier students being able to afford quality extracurricular activities vs poor students.

The SATs and ACTs were the only objective factor a less advantaged minority group could use to show discrimination in admission. It existed as an objective national measure and currently, it's the only ones that we have in college admissions. Frankly, it's the only test we have to measure actual discrimination in colleges. And the virtue signalers want to get rid of it?

Your entire comment is in favor of GT score and SATs. You even said that you were a "dumb 17 year old". You weren't prepared.

> Generally, everybody who entered the military as a teenager notices dramatic improvement on that test when retaking it later life even with no preparation.

No kidding. That's true for everything. It's why you can join the military later in life.

So the military and colleges should take everyone who isn't prepared? All you are arguing for is people should enter military and colleges at different ages because some people are less prepared than others at certain ages.

I don't know why you are so hung up about SATs, ASVAB, only. Why not grades? Why not extracurriculars? Your argument applies to those as well. People mature at different ages. So lets get rid of every measure and let anyone in.

Everyone is against the most fair and objective test measure. There are so many media articles and social media spam about it. Not so much about legacy admissions and the real privileges.

> So then let those people reapply for college when they "mature".

Why not let all people wait and apply once they mature? Why single out “those” people without an artificial unearned advantage?

> Why not let all people wait and apply once they mature?

Yes that's my point. Let people keep trying.

> Why single out “those” people without an artificial unearned advantage?

Who is singling out anyone? If anything you are the one singling people out.

Your comment makes no sense. What are you arguing? That everyone should wait for everyone else? If so, then that's got to be the dumbest thing I've ever heard. Think about what you are saying. So if you get a driver's license, you think you should be forced to wait until everyone else pass their driving test before you are allowed to drive?

Just because you couldn't pass some military test, everyone else should wait to until you can pass?

Talk about being selfish and wanting an "artificial unearned advantage".

The peers are of the same age group (largely), so as a comparative tool within a cohort, would it still not be relevant?

No because the cohort are children operating under artificial constraints that are no longer present once those children move away from their guardians.

> I went to an "elite" Indian college. Anecdotally, I can say the quality of education was quite a bit higher than what could be had at non-elite colleges. I can also say with reasonable certainty that the large salary difference is mostly due to signalling.

The same is very much true for "elite" universities in the UK. The education is genuinely better, but the amount of extra weight given by employers is disproportionate.

Re signalling - apart from simple education, going to an elite college also gives softer skills. For example increased confidence, social skills, networking opportunities, training in etiquette and behaviour or manners, and better alumini connections which could be utilised later on. Elite colleges may have better extra curricular activities too, a better gym maybe, or better equipped clubs, enabling students of the same intelligence to have other benefits than just education.

Now, here's the kicker. Signalling is not about a better education. These soft skills are the primary thing that is being signalled. The signal is: "going to an elite place produces better more rounded individuals".

You can go to an elite school and not gain any of the actual softer skills and stay in your room all day but you would still benefit from the signalling!

> social skills, training in etiquette and behaviour or manners,

How do schools give these?!

You're surrounded by the children of the elite, who model the etiquette and behavior of their parents. After a few years, you tend to imitate them and that allows you to move in those circles effortlessly. That gives you access into high powered careers like high finance, strategy consulting, and corporate law where upper-middle class behaviors are required. McKinsey just isn't going to take someone with the behaviors of a yokel and put him in front of a F500 CEO even if he's a literal genius. You have to look and act the part. That's the true value of an elite education: a rubber stamp and an understanding of the behaviors and habits of the elite.

There is a caveat that this article doesn't consider.

In India, if you are studying for elite colleges, in this case the IIT's, you write the entrance exam designed specifcally for that, which is IIT-JEE.

Most people go for separate training camps to study for JEE. The 12th grade exams are a cakewalk compared to these exams, and the stuff you study for the JEE is sometimes super advanced, like calculus in super advanced levels.

Most kids studying for JEE ( I was one of them), don't optimize for 12th grade exams because the JEE exam rank is all that matters to get into these "prestigious" institutions.

So a bad 12th grade score is by design more than anything.

Though there is definitely a lot of signaling, the research has ignored a very important behaviour.

I’m not sure I understand. This study doesn’t really seem to distinguish between what the economist says it’s distinguishing.

The economist appears to be arguing that this study indicates that elite colleges lead to higher wages due to signaling and not the quality of education.

However, the study basically compares the wages of people who scored about the same in the standardized exams after separating those who made it into the elite colleges and those who didn’t. The ones who made it into the elite colleges appeared to make more money than those who didn’t.

So basically, this study tells us that going to an elite college helped someone make more money than someone else who had about the same intelligence (as defined by their standardized test scores to enter the college) but went to a non elite college.

However, unlike what the economist is claiming, this does not tell us whether this effect is because of signaling, or because the quality of education is better. The economist claims that the study indicates its signaling, but the higher earnings could very well be explained by better instruction in the elite colleges as well.

I also suspect those who went to elite colleges feel far more pressure to inflate their reported earnings relative to those who didn’t.

Recruiters want to optimize for success (because it means they’ll get commission) so they rely on easy low-effort proxy signals like school name and referrals within the company to help filter who they engage with during their limited time for the hiring process. As a broad rule of thumb, it works even if there are false positives and false negatives.

Exactly, they don't show how these people performed in a standardized exit test _after_ college, so there's no way to measure how well they did relative to each other in terms of education.

No, I believe the study shows exactly that. According to the article india apparently has standardized exit exams in many schools.

Well I am from India and did not take any standardized exit exam when graduating. We do have standardized entrance exams.

> I believe the study shows exactly that

No, the article itself says the exit exam is for Secondary School, i.e. High School.

They have exit exams both post high school and post college.

“India is atypical in that its college students have to take standardised exit exams. ”

The first figure has high school entrance exam as the x axis and college exit as the y.

I see lots of complaints in this thread that the article is inaccurate, but this is what it says.

There are standardized exit exams among colleges affiliated to a single university.

Exit means exactly after.

Wrong, the paper says they compared exit scores for universities which have common exams, but different (public vs private) colleges.

There are also non-academic success factors to consider in earnings over time. Family connections and wealth, personal networks, personality, tenacity, etc.

The study being referenced compares college exit exams as well as mean salaries. They found that the exit exams were not significantly different between people who went to elite vs non elite colleges. This doesn’t prove but might suggest that a better education is not what explains the higher salaries.

Or the thing that the test measures is not the thing that causes higher salary. Test does not measure totality of education, just part of it.

Also, quality of education you got is not measure by salary you get after.

There ain't no such thing as standardized exit exams for colleges - especially for the elite rung of the colleges. My college department's professors set their exam papers as did every other faculty in similar colleges - so it's really puzzling what this article is arguing.

Lower rung universities do have papers set by the university for all colleges affiliated with that university but again every university sets their own exams.

There are also entrance exams for MBA schools which are entire different beast and not really connected to what you study in Bachelor's

(Source: studied in the said elite Indian colleges)

> The study being referenced compares college exit exams

How can you have a common exit exam for people who have studied different degrees? No two degrees cover the same content.

Is it covering baseline content that you could expect anyone to know no matter what degree they did... well then I guess that explains why everyone would get the same score!

The difference in quality of preparation of students who study to get into the top universities (IITs, NITs, BITS, handful of private/local universities = ELITE) so far outstrips the preparation of your average engineering candidate, that it is not even funny.

To study computer science at an IIT, you have to be in the 0.01 percentile (1 in 1000). Most of my cousins with sufficient wealth, opted to go study Math/CS at Stanford, MIT, etc. instead of going through the grind of getting into an IIT. That's how difficult it can be.

High school exit examinations are a joke. I studied for less than a week for them. I studied for 2 years (8+ hours per day) for JEE, and I was considered one of the 'lazy ones who started late'.

> In short, elite colleges probably do offer their students more value than their competitors—just not in their lecture halls.

This is an outright lie.

The difference in education between the ELITE universities I mentioned above and your average engineering college is that of Harvard vs a Community College. My brother goes to a well regarded sub-elite university, and the drop of quality from the ELITE and his is mind-boggling. Most IIT professors have PHDs at US top 20 universities. Most sub-elite professors are teaching because they couldn't find a job in CS and have never published a paper in their lives. (The delta is 10x worse for any core engineering branches such as Mech,EEE,Civil,etc)

I would have loved for it to be different. It is entirely unfair that the state of a person from age 15-18 leads to an opportunity gap that's this massive. But, that is the truth of how things are in India.

> It is entirely unfair that the state of a person from age 15-18 leads to an opportunity gap that's this massive.

I was pretty lazy at that age, and I know that if I had applied myself a little bit more, I'd be way further ahead now (in fact, the way things ended up going in the 90's, I'd probably be comfortably retired by now if I'd been a bit more academically successful when I was a teenager). I'm not bitter or upset about it, though - I feel like I've gotten what I was willing to work for. I'm watching my own, much more disciplined and motivated, son grind away through his junior year of high school trying to keep his perfect GPA so that he can get into a top college and I can't help but worry that any attempt to make things "fair" would disadvantage not just students like him (potentially, anyway) but everybody: if there's an upper threshold of effort beyond which additional effort is ignored in the name of fairness, it becomes impossible to distinguish the really committed from the "just showing up".

Most IIT professors have PHDs at US top 20 universities. Most sub-elite professors are teaching because they couldn't find a job in CS and have never published a paper in their lives.

What does that have to do with the quality of teaching? I feel like you might be falling into the trap of equating "quality" with "eliteness".

It is entirely unfair that the state of a person from age 15-18 leads to an opportunity gap that's this massive.

Agreed, with respect to the US. We would all be better off if, instead of a perfectly scheduled ladder from age 5 to 25 of school, university, and career, it was normal to have a wide range of ages in all university programs, at all tiers. You should be able to decide at 28 or 38 or 58 that it's time to take the qualifying test and be admitted to a prestigious school.

- You studied only a week for high school exams because you already knew most of the material having studied for JEE.

- The "just not in their lecture halls" seems like extrapolation by the journo (I agree with you otherwise). I cant find the pdf of the paper but I doubt the researcher is making that claim. Perhaps it means that motivated students "learn" despite poor teaching at the non-elite colleges.

edit: Ok someone here has posted a link to the paper. The paper is not very good. They arent comparing IITs vs run of the mill colleges. They are comparing public and private colleges within a university which has a common college exit exam. Afaict the university is not even mentioned.

>The difference in education between the ELITE universities I mentioned above and your average engineering college is that of Harvard vs a Community College.

A lot of people (myself included) would say that the difference between Harvard and a community college also resides entirely outside the lecture halls. I can't speak to Harvard, but my experience with community college and a well regarded University of California campus was that instruction was much better at the community college. No one takes a position at an elite US university to teach. They go there to do research. Conversely, the professors at community college don't generally do research, so they have much more time for their students.

I have taken classes in Physics, Math, and CS in a top 20 ranked US university and a university ranked in the 200s. The difference in teaching skill and the difficulty in the materials taught for the same course is massive and very noticeable.

This is not even to go into the research that the professors do and the academic levels that the students are in. The difference is huge.

>The difference is huge.

Which way? My experience was that my (not quite top 20, but close) university professors were an extremely mixed bag. None of them were half as good as my best community college professors, and some of them were downright awful. I would be truly shocked if you told me that lower ranked universities were worse.

For one, courses were significantly more difficult and the homework was harder in the top 20 university. For example, the junior level CS algorithms course I took at both were so different that I wouldn't even call the one I took in the 200s ranked university an actual algorithms course. This was just the worst case though; the other classes were better than this class but the differences were significant.

I agree though that it can be a mixed bag in that there were good and bad teachers in both universities but I found the top 20 university had in general professors who consistently taught better, the courses were better structured, and the professors seemed more motivated to perform well than the 200s ranked university. There were a number of exceptions on both sides but the exceptions favored the top 20 university.

The so called exit exams(tests) in Indian colleges may not be what Westerners expect it to be. In Indian exams, challenging problems are not touched upon at all. Exams are about how much ground can be foraged rather than how deep you can wade in. You can easily get by through cramming and rote learning. I am sure university/college/school exams still lack questions that force students to solve problems, applying learned concepts along with their analytical skills.

There are two types of exams in India:- Entrance exams - where the grasp of concepts, analytical ability etc are tested. School/college exams - where it is all about regurgitation of information.

With latter, the only aptitude that can possibly be assessed is the tolerance for a monotonous and brainless desk job.

A funny anecdote. Our Visual Studio C++ professor fell sick so another prof took our practical exams

These were the questions:

1. Write an even number program 2. Odd number program 3. Palindrome

Not an elite college, mine.

It's similar in the US, where we have the SAT IQ test and the GRE (SAT repeat plus basic knowledge test in a subject area).

Neither has the sort of deep knowledge you'd see in a US Graduate Degree "qualifying exam" which tests depth of understanding of undergraduate material (for which many students use remedial graduate school classes to prepare)

This is broadly true, but qualifying exams can be wildly different even within faculties, let alone across schools - enough so that I can't imagine a meaningful comparison of "results". They are also often pass/fail.

The article talks about how students who went to elite Indian collegs perform similiar to students who went to low tier colleges in standardized exit tests wich is apparently a thing in India.

The only issue is, students don't take any standardized exit tests in India. Which makes the whole paper pointless.

I went some digging to see what data this paper is using. Apparently this is what I found out.

> All students in colleges (private or public) affiliated with the same university take the same exit exams. These exams vary by stream of education, but conditional on the stream, private and public college students study the same curriculum and take the same exit tests.

> I obtained administrative data from colleges in a district in North India and the university with which they were affiliated.

The issue with using this data is that almost none of elite colleges in India are affliated to some University. They are their own University. So apparently the data of the paper doesn't include any elite colleges! Kind of ironic given the title of the paper "Prestige Matters: Wage Premiumand Value Addition in Elite Colleges"

> students don't take any standardized exit tests in India

From the article:

> There, pupils in their final year of secondary school sit a leaving exam known as the Senior Secondary School Examination.

(Emphasis mine.)

There is a bit of a mistake here. There isn't any one Senior Secondary School Examination, but many. Some of them are standardized at the state level, and I know of two which are standardized across the country, the largest of which is the CBSE. I assume this mistake is from the Economist, since the abstract of the paper itself says "Admission to the elite public colleges is based on the scores obtained in the Senior Secondary School Examinations" (emphasis mine). Going through the "data" archive linked on the paper's page, I see references to the CBSE, which makes it likely that that was the standardized test the author considered. The CBSE is, by definition, a standardized exit test for secondary school students in India.

> am actually amused this article was based on an research paper published by Universitry of Virginia

The author of this paper is of Indian origin.

I was referring to the exit test scores that is used by the author for comparing students who goto elite vs normal college. See my updated comment above.

This is an interesting topic and has many analogies.

If you gave mathematicians some general exam and have all universities participate, I would also not expect top universities to score the most. However, if you asked instead what the money value is of their research, then MIT, Oxford, et al. would score highly.

I don't want to extrapolate on the economist.com article, and whether it's true or not, but rather just illustrate that this happens in other contexts.

Another example is salaries. Are the best programmers in the areas that pay the most? I think no, but the ones whose work make the most money for their employers probably are. A case in point is the alpine Emacs coder that featured in a comment a few days ago, Thierry.

The main issue is that if you focus on the money part, then you focus on ideas that have settled. Many small places or small universities are still at the distillation step and the distillate tends to move to where the money is. I use this argument as one of the main arguments for fundamental research, and also for why it has cognitive dissonance with earning potential.

Well there's the Putnam competition, which MIT has been completely dominating recently (https://kskedlaya.org/putnam-archive/AnnouncementOfWinners20...).

Are these tests like the SAT? I studied pretty specifically for it before college. I imagine I'd do worse after college because my degree didn't have much to do with the tricky algebra questions or weird vocabulary that it seemed to focus on.

I don't know. One difference between k-12 and college is that we are much better at long-term remembering things we learned earlier in life.

Some details from the paper: - The paper studies colleges that offer a three-year undergraduate course in arts and science (no engineering or medicine). So, extrapolating this for IIT/NIT vs non-IITs is just argumentative with no data points.

  - The statistical data did not survey students who managed to secure a seat in an elite public college using reservation quota.

  - The sample data is from admission cohorts a couple of decades ago (1999 - 2002).

  - The detailed in-person survey was done only in urban areas during 2011-2012 time frame.

  - Data sample:
    * Initial survey request sent - 1981
    * Actual surveyed - 1506 (76%)
    * Either self-employed or salaried - 748 (49% of surveyed)
    * Final data sample - 439 (58% of employed)
      # public colleges - 190 observations
      # private colleges - 249 observations

  - The entire analysis (and the claim) is made comparing the outcomes of students at the margin of admission (around admission cut-offs). So, if a student secured top score and entered into a public college and gets a top dollar salary, the author weakly attributes it to better learning environment.

Can the exam be the confounder here, where the exam is just not good at differentiating the different in educational experience between the universities.

Second the job interview and exam might just be looking for skills on different dimensions ?

Yes that is a possibility in this sort of research design. The exit-test scores are obviously taken as proxy for educational quality and the study cannot test for any "secret sauce" beyond that.

On the other hand, there's only so much one can do with observational data doing these sort of causal analyses. I'd say its still an interesting outcome.

It’s also possible that tests are simply a very poor way to assess knowledge/aptitude. I certainly felt this way when taking the SAT/GRE in the US at least.

Am I the only one that finds their primary data plot at the top terrible? Journalists should have a look at the Grammar OF Graphics .

Who you know, not what you know...

Disclaimer: I graduated in CS from one of these "elite" colleges. I also took the JEE in 2013 when it underwent significant changes.

Back in 2011, the government decided to revamp the educational grading system by introducing "Continuous and Comprehensive Evaluation" (CCE) for class X. We were the first batch to go through it and every assignment/internal exam had some weightage in your final grade. Fast forward to 2013, there was a repeated attempt to bring similar change to this system. The central government merged the AIEEE and the IIT-JEE into two sets of exams - the JEE Mains and the JEE advanced. They also introduced weightage to your XII grade marks in the final rankings - meaning you had to do well in your boards in addition to the one-day exam.

Admittedly, this was less radical than year-round performance, but it was still significant enough to tell everyone - "Hey, you need to perform all across our evaluations, not just in one exam". This meant that you could have gone through all the folks who took the CBSE XII exams in my class - and they would have been in the top 5% (maybe even top 1%).

Having said that, at the end of the day - apart from signalling, the main reasons to get into the IIT/NITs were infrastructure, the environment and the network. The infrastructure was way better than a smaller, private university and your entire class is filled with the top people from your batch.

Even if we conceded that the curriculum and the faculty were not at par with the top universities in the world, the talent surely is. The extra-curriculars and the competition within ensured that you constantly honed your skills to remain relevant - we took MIT OCW courses, participated in global hackathons etc.

TL;DR - High-school test scores were relevant when it mattered in college entrance exams. The moment the weightage was lost, the relevancy was lost.

There are no standardized “exit exams” for Indian universities, unless one is talking about those colleges affiliated to a specific university system (but those are not referred to as “elite” institutions in common parlance; just terminology, I’m not making value judgements). Many states in India have a 10+2 education system where the “+2” (last two years of high school) are referred to as “junior college”. And admission to “elite” institutions is through specialty purpose entrance exams which are typically considered to be much harder than high school / junior college exit exams. Evaluation in the latter can also be a crapshoot, so scored in the high school / junior college exit exams might not have a very strong correlation with scores in the university entrance exams.

I don’t understand what’s actually being analyzed or commented on (both the Economist article and the source journal article are paywalled for me)

Like most things, it’s not what your education makes of you, but what you make of your education, specifically your ongoing self-education.

I dont disagree with the premise that credential based tertiary education credits for jobs are pointless and a genius can come out of any university. You can do well despite of where you studied. There are people who succeed in life due to no talent and just connections.

However, this analysis is just based on outright flawed data. The author has no idea about how education system works in India, or deliberately chose to ignore it because this is what could be done on the data they had accumulated. And I am not talking about the disclaimer about data sample being small, the entire model is wrong for majority of Indian students.

Couple of things I want to comment on when you look at the article and the original paper :

1. The study treats few public institutions as "elite" and private institution as for non-elite people. That is wrong. There are private institutions which are elite too. My alma-mater is one of the elite ones and it was not a public university. This is the least problematic error in analysis.

2. The assumption is a common academic score being used to score university students. For college entrants in early 2000s (the time period when the survey respondents applied to colleges), people had to literally give 50 different entrance exams to get into universities. The score which is used to evaluate who was marginally better to get into elite university and who was rejected (the entire premise of the article), IS NOT what is used to decide entrance into universities in India. Its getting slightly standardized now, but was really chaotic 15-20 years back. There is a common exam that people give at secondary / senior secondary level but very few universities entertain them at entrance. Actually there are many "boards" to give Senior Secondary exams as well. Net net, there is very large variety of ways by which universities in India take entrance. Even the elite ones. I think, given the author is from the field of economics, they are overgeneralizing. A few (not all) economics elite universities do treat senior secondary tests as entrance criterion.

3. Exit exams used to compare academic credentials of students to see if they are academically better than the other DO NOT EXIST ! They dont even exist now and there is no point of them existing. Every public/private university in India (at least the elite ones), have their own grading/curriculum. So the academic credentials cannot be compared directly. With different universities having different grading method, people having somewhat similar academic distribution is a known thing (I dont recollect the name of the phenomenon exactly, but its studied and published). That said, it doesn't go with how the data has been modeled here.

Overall, the author's model is wrong for many (50%+) of Indian students. They might be referring to economics colleges (given they come from field of economics) Delhi University, Mumbai University and Kolkata University type of institutions where their model of tertiary education fits correctly, but in that case her analysis doesn't work well for most of "India". Totally wrong messaging.

Please, please Stop sharing articles which requires paid subscription.

https://news.ycombinator.com/item?id=23597733

It's explicitly allowed by the HN site rules.

Why should we expect it to? Education does not increase fluid intelligence.

Uh. Other things probably factor into test scores than that?

You're asking why we should expect college to increase test scores?

<various babble>

This just means "Nepotism is real". Results should carry to all countries, though I doubt that large and present studies are available.

Nothing to see here.

You don't graduate university for test scores. Test scores only work in low level learning like high school.

For this study they used -

"These exams test for language competencies (English and regional language) and stream-specific competencies; for example, commerce students take tests in accounting, taxation, and so forth."

Basically IQ tests.

Maybe they learned more. Maybe it's about cohorts. Maybe elite universities are used to signal. You can't tell from this study.

DOI: 10.1257/app.20140105

Reply on news.ycombinator.com