Hacker News Reader

Comment by lisper | Hacker News Reader

I hate to burst your bubble but unfortunately this just kicks the can down the road. How do you measure merit? Because [EDIT] many measures of "merit" (was: standardized testing) can be (and often are) biased by race, cultural background, and economic status. For example:

Student 1 has built a fully automated chip manufacturing line in his basement.

Student 2 has build a robot that solves a maze in fifteen minutes (against the current state of the art which is a few seconds).

Which one would you admit?

Student 1 is the child of billionaires, and it's not clear how much of the work was actually done by him/her and how much was done by employees hired by the kid's parents.

Student 2 lives in Sudan and built their robot out of locally available materials, in the process inventing a new kind of motor built out of coconut fronds.

Now which one would you admit?

View on HN

It’s as if you’re trying to make perfect the enemy of good. Standardized tests aren’t perfect but they give less privileged kids the best choice at attending a good college.

The kids of the upper class and the rich will always have an upper hand compared to the poor. However, standardized tests limit how wide the upper hand is. An upper class kid still has to study and pass the test, and the poor kid can also do that.

If admissions become “holistic”, poor kids would have little chances. Good luck to that poor kid competing subjectively with kids whose parents send them on impressive charity trips and get them unpaid internships at the most prestigious companies.

I think you are underestimating the extent to which standardized tests can be (and have been) biased.

https://www.nea.org/advocating-for-change/new-from-nea/racis...

That's soft bigotry of low expecations. All you have to do is look at data that includes Asian Americans, which is always conveniently omitted from these racist narratives. Even non-Americans routinely do better on American standardized tests.

It is not. GP is saying that the "expectations" in question aren't as applicable to potential as they're purported to be.

>Asian Americans, which is always conveniently omitted from these racist narratives.

Ironically, so is the diversity of the Asian American community.

To the contrary, the NEA wildly overestimates it and employs junk question-begging "disparate impact" [0] reasoning throughout. The article is full of stuff like this: "There is a clear correlation, for example, between test scores and property values."

To the extent that society is meritocratic at all and intelligence is heritable (and it is), we should expect test scores to correlate with all manner of measures of success, including property values. Articles like this don't even take that question seriously. They just ignore it. It's proof by repeated assertion. It may be fashionable to insist that this is prima facie evidence of bias, but that is a question of logic and not a difficult one, whatever exceedingly average minds like Ibram X. Kendi think of it.

[0] As a legal concept "disparate impact" is what it is. The law means whatever its authors intend it to mean. But as a matter of logic, it's embarrassing, and beneath this forum.

"Since their inception a century ago, standardized tests have been instruments of racism and a biased system."

Standardized tests were invented in Sui dynasty China, in the early 600s AD, as a way of selecting officials for the imperial bureaucracy. They were invented precisely because they were more objective than the prevailing method of selecting officials - recommendations from the aristocracy.

There is a long history of standardized testing being a means for rewarding merit, instead of more easily corruptible methods of selecting officials/students/etc., such as recommendations. Just to illustrate my point: Do you know why Harvard abandoned standardized testing in 1926 as the sole means of determining admissions? Because "too many" Jews were passing the admissions test. Harvard's "holistic" admissions policy was invented for the sole purpose of restricting Jewish admissions.

I think you're underestimating how biased literally every other possible metric of admissions can be.

The entire concept of an “admissions” department was based on the historical fact that too many Jews were being admitted and too few WASPs were. So they included a “character” criteria and fixed the problem.

Now, too many Asians are being admitted based on test scores. Oh no! To fix this problem, Harvard consulted their history department and included a “personality traits” section. Is it any wonder that Asian students scored low on this?

https://www.nytimes.com/2018/06/15/us/harvard-asian-enrollme...

Oh my. This obsession over race is so bewildering to foreigners like me.

Yeah the SAT is the worst admissions metric, besides all the other ones.

What would you use in its place? You think grades don't have bias?

Interviews, transcripts, (optional) test scores, letter of recommendation, a set of common essays across all schools, and an optional supplemental section or portfolio to showcase any personal achievements not covered by the other standard categories.

Oh wait! That exists — it’s called the Common App, and it’s what most private colleges today use, from the Ivies to elite tiny liberal arts colleges with the largest share of students from the 0.1% that you’ve never heard of, like Pomona College.

Transcripts aren't useful across different high schools. Rich kids have more connections for rec letters. Rich kids get professional essay help, and lying kids make up a good story. Kids and parents with lots of time/money/connections on their hands get a portfolio of community service etc built up. I knew these kids in high school with resumes like veteran philanthropists, and it worked.

I think the only good one out of those is the interview.

Essays are incredibly biased though! Do you really think that for some reason essays actually have to be written by the person applying, and can't be gamed with money? That an overworked public school teacher is going to write a better letter of recommendation than a private school teacher? That a rich kid is going to have worse extracurriculars, portfolio or achievements?

The Common App is great, but it's not magically less open to bias than standardized test scores.

> That an overworked public school teacher is going to write a better letter of recommendation than a private school teacher?

LOL. My understanding is the really good prep school college counsellors golf with one or more high-up folks in elite university admissions offices, and get the inside scoop on exactly what they and their peers in other universities are looking for in any given year, such that they can even tune an essay or letter of reference for a given school based on that non-public information and advise students which schools to focus their application efforts on, based on their background & activities.

Not only that, but the person reading the essay is also biased and will select for students who align with their bias. It is a terrible admission metric.

It’s naive to think schools don’t have a system in place for this: separating piles into buckets of test score, ordering by grade, marking a certain number from each bucket as worth another look, then ordering by essay, marking a certain from each group, and repeat on any other metric.

Many schools, selective or not, actually do this whole process — multiple times, with each admissions agent doing a separate order of criteria, to ensure everyone’s application gets read at least twice. The idea being that those with the most “let’s give them another look” across the board are the most notable. Then from that shortlist the debates comparing each applicant, usually sorted by geographic proximity to each other, begin (at Harvard, if you’re from Texas you’re not really competing against New Yorkers for a spot, you’re competing against other Texans for the XX number of Texan spots they usually admit a year).

I did a short stint as a student worker in the admissions office of a very selective college in California (<5% admission rate, but not one many could name off the top of their head), and this is more or less how it worked

They use it because it allows them to ignore standardized test scores and just do admissions based on their own preferences. They used less merit-based metrics because they don't WANT a meritocracy.

Question. Which tells you more about a student:

1. An essay that was probably written by chatGPT then edited by the students parents

2. A test taken in a supervised, controlled, timed environment

1. Would absolutely not get you in anywhere selective

2. Would absolutely not either

I believe the whole concept of “meritocracy” for purpose of admissions is a lie— choosing the criteria to measure against is itself a subjective act.

> I believe the whole concept of “meritocracy” for purpose of admissions is a lie— choosing the criteria to measure against is itself a subjective act.

Let me ask you. Do you also believe that requiring a display of proficiency in mathematics to get into the best schools is inherently discriminatory? What about requiring a demonstration of the capability to understand and complete basic subject matter material in the fields or reading, writing, or scientific literacy?

There’s no shortage of people who meet any of that criteria!

The whole point of a selective college is they have to select from a pool of already qualified applicants. There is no objective measure to measure against when you’re splitting hairs. Even were you to limit it to “objective” requirements like test score and GPA, how do you decide between two students for one spot when both have the exact same scores?

There is no shortage of perfect scores applying to Harvard. And yet a majority, or even a plurality, of any given class of admits didn’t have perfect scores.

There is no shortage of perfect scores applying to Harvard.

1,000 people typically receive a perfect SAT score yearly.

> letter of recommendation

We've gone full circle.

Is there literally any test in which:

"Students of Color" receive scores in the same distribution as white/asian/hispanic students (same fraction of 1s, 2s ... 35s, 36s on the ACTs for example)

AND

Top scores are meaningfully distinct from the population average? (because the first condition can be trivially fulfilled by having everyone score the same)

What is the evidence that the tests now are biased other than that different groups score differently?

When your history ensures that some groups are socioeconomically disadvantaged, every possible method will be biased. It's unavoidable.

So why don't we focus on socioeconomically disadvantaged individuals instead of focusing on racial metrics?

With affirmative action gone, that's pretty much exactly what's going to happen.

Everyone understands that standardized tests are biased. They are still the least bad way to identify students from underprivileged backgrounds who have high potential to succeed in college.

I am someone, and I don't understand why they are. Do you have a primer handy on this subject?

SAT scores are correlated with wealth. This CNBC article is a decent introduction to the issue, but be aware that anything you read on the subject is likely to be pushing a particular narrative so it's tough to find a neutral primer anywhere.

https://www.cnbc.com/2019/10/03/rich-students-get-better-sat...

It’s funny how that article doesn’t mention the most obvious reason - it turns out intelligent people generally don’t get jobs as janitors, retail workers, or other minimum-ish wage jobs. Those intelligent people then have intelligent babies.

For some reason a large part of our society doesn’t want to acknowledge that genetics (highly) influence intelligence.

Anyone who has lived in the real world knows that the conditions in which you grow up have a massive effect on where you end up. There are entire communities of people who live in deep poverty. It's not because they're all - uniformly down to the last person - dumb. The parents are poor, don't have a high-quality education, work long hours, can't afford tutoring, can't help their children with their coursework, etc.

There are entire countries trapped in poverty. China was desperately poor until just a generation ago. Did Chinese people suddenly become smart for some reason? Which is the next country that will magically go from being filled with dumb people to being filled with smart people?

As I said, just a bit of experience with the real world and critical thought will show that your explanation is nonsense.

I wasn't talking about different countries. Obviously there are issues that can hold large geographic regions down.

(Basically) no intelligent teenager in America is held down by poverty. My parents were relatively poor, didn't go to college, worked long hours, and I went to a public school and certainly didn't have any tutoring and didn't receive much help with homework.

I did well enough on my ACT to go to a private college for pretty close to free. Even if that doesn't happen you can go to a state school that doesn't need great grades, take out loans for the entire amount, get a degree for a good paying job, and suddenly you're out of poverty.

> Did Chinese people suddenly become smart for some reason?

Yes, they stopped following strict communism. They pivoted to authoritarian capitalism and got wealthier.

thank you. I read that article and the linked NEA one. Lots of info on biased roots in history, not much information on the mechanisms that create the bias or unbiased standardized tests as an alternative.

I would think there would be a lot of demand for an unbiased standardized test. All I saw for suggestions was to use GPA instead, which clearly has a lot of bias potential... is there a recommended replacement for the SAT/ACT?

That’s an awfully big article with no actual examples of how the tests are currently, in any way, biased.

They're not biased at all. People don't like them because they're accurate.

This is so naive. Poor kids do worse in school because their lives lack the material and parental support necessary for quality education.

Yeah I see this take all the time. Admissions offices are allowed to take a student's means into account! So in deciding between two students with equal SAT scores, one from a Greenwich private school and one from South Tucson, the one from Tucson is the more impressive student.

Affirmative action was misguided because it assumed that because one student was Korean and the other Mexican, the Mexican kid must be disadvantaged. Besides the fact that there's an inherently racist worldview baked into that, Newsflash! There are tons of poor Korean kids and rich Mexicans!

Why is SAT score a good way to decide which student is more deserving of a spot?

There should be some SAT score floor. But beyond that other factors should take over. If the floor is at n and two applicants appear - one with a score of 1.1n and the other 1.2n, I don’t think that’s enough information to decide who should get the spot.

There is an incredible about of SAT prep that is free.

It's a good way of figuring out who is intellectually capable. SAT scores have a very good prediction rate of success in college.

High SAT scores are only a good predictor against low SAT scores. Can you say that extremely high SAT scores are a better indicator of college success than very high SAT scores?

I’m not saying SAT scores are useless, just that they should only be used as a filter and not for ranking.

> Why is SAT score a good way to decide which student is more deserving of a spot?

Name a better way.

SAT score floor + lottery?

What makes you think that is better?

Vox had an article about college lotteries earlier this year you might find interesting:

https://www.vox.com/future-perfect/2023/4/19/23689402/colleg...

Lottery is a metric that can't be gamed, if implemented correctly. It may not ever be the best system, but it also can't end up being what was a better system that was gamed into a worst system. It provides a certain level of consistent mediocracy between various other systems which rise and fall as they are gamed.

what are the other factors?

Spot on. I don't see why this is so hard for people to comprehend and why they are fighting the obvious. That very basic level of nuance tends to be missing from these conversations.

> I don't see why this is so hard for people to comprehend and why they are fighting the obvious.

It's because these people only see the world through the one-dimensional lens of skin color.

Further, it's an attractive way to view the world when you're a complete idiot with nothing of substance to offer society. It leads these people to infecting society with parasitic and fallacious ideas that you see manifested in the extremes of both political parties.

Magic that happens in top universities in US is combining money (kids of billionaires) with smarts (kids with 1600 SAT) in one place. Both bring different skills to the table and result is disproportionate share of top scientist, business and political leaders produced by those universities (from both classes of people). Removing either group from university will just lead to university stopping being elite. Considering that number of billionaires is measured in hundreds (so only a dozen or so kids of billionaires enter universities every year), university may just admit that one kid.

On Student 2 who build something in Sudan from stick and rocks. Unless he is from elite family he most likely did not get proper school education and won't be able to keep up with rigors of studying in top university even if he is very smart. Harvard is not really in a business of providing remedial education. With that if he is really smart and resourceful, he had a very good chance of doing very well for himself in Sudan (becoming entrepreneour, building soemthing local, become warlord, etc) and then his kids will be fully equipped to go to top university.

How is that an example of standardized testing?

Good point. I've edited my comment.

I'm confused here. Both facts are salient (what was built and under which circumstances was it built) to any discussion of merit. The main complaint I see here is that admissions committees should use as much information as possible, which I doubt anyone disagrees with. What is racist is someone saying 'Oh, student 2 is black, thus without any further information, I'm going to assume he's poor and from Sudan'.

Case in point, we had a very wealthy black student in my college. This woman was not disadvantaged in any way, yet she played the race card all the time in order to claim a disadvantaged background. I'm talking about a family that would take their kids to France and England to summer. That level of wealth, yet framing all her accomplishments as if she came from the inner city. That's disingenuous, yet the (now-gone) affirmative action camp would have gladly taken her checking the 'African-American/Black' checkbox as a sign that all her accomplishments should be judged on a poor disadvantaged upbringing. How is that not racist?

I hate these types of arguments that create an extremely contrived example. If we’re making a choice between these two students, then you can’t really go wrong. But that’s not the choice being made.

> Now which one would you admit?

Why not both? I think a lot of the debate over college admissions misses the fact that so much of this is artificial scarcity. Some of the big schools like Harvard/Yale/Princeton etc. could easily increase the size of their incoming classes many times, and still have them only be filled with highly qualified candidates.

Top schools exist not just to educate, but to ensure that the social hierarchy is maintained. If it were purely to educate or to ensure diverse learning environments, the top Ivies could solve this easily by quadrupling the size of their classes, but then this would of course dilute the exclusivity that is the primary reason for these institutions in the first place.

We'll never have a perfect way to measure merit, but that doesn't justify the status quo. It would likely reject both students for a third student that checks the right intersectional boxes, even if coming from a more privileged upbringing than the other two students.

We should be constantly improving our ways of measuring merit, not throwing up our hands and pretending it's meaningless to try.

Is this a need-blind or need-aware institution?

If need-aware, does the admitting class have enough full pay to cover the costs of those needing scholarships (wouldn't want to actually tap into that tax-free endowment)? If they need more full pay, then S1, if there are already enough full pay, then S2.

Student 1. He has the resources needed to move ahead. And unless my university can provide self help to the motivated, student 2 may not work out well here.

What kind of child, even that of billionaires, wants to build an automated chip manufacturing line? Maybe in Minecraft. Or in Roblox.

I believe that GP would admit the child of billionaires. People who promote the meritocracy myth have an agenda and are sticking to it. Do you really think you can change GP’s mind with logic?

S1 definitely.

depends on their race

standardized tests are not biased.

Well, I'm getting down-voted even though I'm correct. This is one of the problems in our society generally -- people have decided that what they think is just must be correct because they assume that life is fair.

Reply on news.ycombinator.com