Hacker News Reader

Comment by logicx24 | Hacker News Reader

This, like every other similar attempt made to "equalize" the SAT, is misguided and will accomplish little.

If the SAT correlates too strongly with wealth, then Collegeboard should make a better test. Make one that changes significantly every year so direct test-prep is hard. Choose different types of critical reading passages and questions each year, vary the style of the math questions - make the test different enough each time so studying past tests isn't valuable. Then, if you eliminate the advantage direct prep gives, the correlation to wealth should weaken, and the test should get closer to measuring aptitude.

But instead, Collegeboard continues to shoehorn political objectives into an already broken exam. This is a mistake.

View on HN

Imagine a perfect SAT that preparation can't affect; for the purposes of the hypothetical, the test just stares into your soul and measures pure scholastic aptitude.

If (a) scholastic aptitude has a significant effect on your lifetime income, (b) people tend to marry people of similar social and economic class, and (c) scholastic aptitude is fairly heritable (through genes, environment, whatever)...

... then you would expect the Perfect SAT scores to correlate pretty noticeably with family wealth. This wouldn't be a sign that anybody is doing anything wrong; it's just as natural as water flowing downhill. And these are reasonable premises, with strong empirical evidence for each of them.

So, question: just by looking at the correlation of SAT scores with family wealth, how can we possibly tell how broken the test is? How can we know how far the real SAT is from the absolutely un-gameable Perfect SAT?

I mentioned this below, but if the test is a perfect predictor of future success, then the people that perform well on it are absolutely worthy of college admissions, and there should be no further analysis required. If the test is perfect, then the characteristics of the people that do well on that test don't matter. The reason we need to do implement equalizers like this is because such a test is impossible.

Because we've done studies to address exactly that question. What we've found: The least-gifted children of high-income parents graduate from college at higher rates than the most-gifted children of low-income parents, hence the adversity score to rectify the inequity in life circumstances.

https://www.independent.co.uk/news/science/money-academic-su...

This study has nothing to do with the SAT. Clearly, money is required in order to study full-time in the US, both for tuition and living expenses, so people with more money graduate more often because it's easier to not work during school, and to pay for tuition. But none of that is relevant to the point you replied to, which was that even a perfect test would have results that correlate with wealth

> The least-gifted children of high-income parents graduate from college at higher rates than the most-gifted children of low-income parents, hence the adversity score to rectify the inequity in life circumstances.

Looking at graduation rates instead of admission rates is problematic. Less gifted and underprivileged people also graduate at lower rates despite being admitted using affirmative action procedures as the College Board is now implementing. This means they do not have a degree that would advantage them, but are saddled with undischargable education debt that puts them at a severe disadvantage for the rest of their life. The system not only continues the inequality but makes it much worse. If college was free or education debt was dischargable in bankruptcy then it would not be the oppressive system it is currently where people unable to succeed are admitted, then forced to pay back debt they can not afford after they fail to graduate. Only a sadist and a racist would support such an inequitable abusive system.

So we need to correct the SAT so that the students most likely to be doomed to fail will be admitted to higher education?

To me it doesn't sound like you're trying to make the SAT reflect say academic potential. It seems like you're just trying to inflate the appearantly already inflated scores of the disadvantaged. Who will proceed to not get scholarships, fail out, and be saddled with debt they can't pay off or be discharged. This of course is the road to an egalitarian society!

Who needs to bother with difficult things like making the disadvantaged do a little better in school? Just add 100 points to their SAT. Problem solved.

Once you combine all those premises, the "pretty noticeable" correlation might be something like ten percent. So the problem of it correlating too strongly with wealth would be solved.

> combine all those premises, the "pretty noticeable" correlation might be something like ten percent

it might be, but it will probably be significantly higher.

Your suggestion relies on the assumption that there is some way to change standardized tests such that they do not correlate with wealth. Perhaps the problem is that academic ability is directly correlated to the kind of test prep you receive, so unless you set a test paper full of trivia questions the student that has gone through test prep will perform better every time.

I do not agree with this idea either, but only in that the adversity score will be opaque to students, so there is no recourse if it does a poor job of addressing the current issues with college admissions.

If you do set a test paper full of trivia questions, that just changes the sort of things in the test-prep program. Anything studiable is going to correlate with the amount of time spent studying, therefore with wealth. People do study for Jeopardy.

> Your suggestion relies on the assumption that there is some way to change standardized tests such that they do not correlate with wealth.

Good IQ tests exists that are hard to game. They are easy to learn easy to vary and easy to measure.

(I think I have heard someone here talking about IQ tests that include language and geography questions. This is not the kind of IQ tests I'm talking about. Perfect IQ tests should measure your processing speed, not your "software" - i.e. education.)

The SAT is supposed to be a test measuring scholastic aptitude not a pure IQ test.

IQ tests always have a knowledge component anyway. Unless you specifically design a test to only measure something like working memory. And who cares about that. Colleges would only care to the extent that it correlated with performance.

But you can have 2 people who have equal performance in a given field even though one has a higher working memory because the other has better context from more time spent reading.

> Unless you specifically design a test to only measure something like working memory.

The ones I refer to typically test what I'd call pattern matching / pattern synthesis by presenting multiple choice questions showing a number of patterns and asking which out of several patterns comes next.

You can teach how to solve it to a schoolkid and it is still really hard to practice for.

You're talking about Raven's Progressive Matrices and variations thereof.

RPM only measures a very small subset of what we normally thing of as general intelligence, and they correlate with general intelligence less strongly than vocabulary. They definitely aren't nearly as good at predicting future college performance as the current SAT is.

>You can teach how to solve it to a schoolkid and it is still really hard to practice for.

It's actually not, you can definitely practice for them. And they are very sensitive to repeated testing.

Here's a cited Psychology Stack Exchange answer with a good summary: https://psychology.stackexchange.com/questions/20177/does-pr...

RPM was developed to be free from cultural bias, but we now know that this isn't the case, they can actually be more biased than verbal tests depending on the culture.

Hypothetically, what if it was true that wealthy people's exposure to better education over a lifetime, reduced stress distracting them from learning, better and more abundant roll models, etc, meant they would do better on pretty much literally any test you could put in front of them?

Most research points to this being true.

Then wealthy people show more "merit" and the test is working as designed. If the test is an accurate predictor of success in college and success thereafter (which is a big if), then that people who do well on it are more deserving of college admissions. The ancillary qualities of that group of people (wealth, race, etc.) are irrelevant.

This is the reality of the situation that people don't want to admit. This is a social engineering problem related to the outcomes of college entrance processes. It's not really about the test or how we are measuring aptitude.

This problem would be far better solved elsewhere in the chain instead of blurring the meaning of data and pretending that things are different.

Yes but fucking with the metrics costs almost nothing. Does almost nothing too.

It can even backfire by sending a bunch of disadvantaged kids to programs they're not prepared to succeed at to "help them" get saddled with college debt they can't discharge.

The SAT is (well, was) supposed to predict academic success at a college/university. The idea being that an institution could admit students who would likely graduate, as opposed to students who did not.

It's heading away from that.

The student who spends time preparing for a test, reviewing the material, studying previous tests, drilling with practice tests, actually has more scholastic aptitude than the students who do not do this because these are actual research and preparation skills that are valuable to success in mastering college material.

The only skill you gain while cramming for the SAT is how to cram for tests like the SAT. The gameability of the SAT is not an indication of general scholastic aptitude

If general scholastic aptitude measured something other than how to study for tests and write essays, then you might be onto something, except that college is simply more of the same.

Sure, maybe if you went to a degree mill.

I always wonder where people get this view of college (the GP's view that is). I went to a non-name state school for undergrad and it was nothing like the above.

You have just described what college is.

College is all about studying/cramming for tests.

And yes, this applies to "top" engineering colleges and otherwise.

Maybe for you?

It is for a whole lot of students.

Sorry to hear about your poor educational experience.

I means, it's the educational experience of top colleges, such as Carnegie Mellon, that I attended.

This includes the engineering, math, CS, and social sciences class that I took.

I went to Stanford and yeah much of the time was learning the material, then preparing for tests by reviewing material and studying. There was also doing projects, researching, and writing papers arguing various positions and analyzing things. As well as some novel research.

It's very strange to hear people here saying all these things are dumb or foolish or something, and good colleges are not like this at all and this is a "poor educational experience". OK, well, what was their school like then? They don't say. Details and names of schools are omitted from the posts of the critics. Are they talking about the drinking and sex at party schools? Or maybe the football games they attended? That is all fine but I don't usually include that when talking about the academic experience per se. Right, studying, learning and preparing for tests is not about those things, true. But so?

Sorry, you don’t see the distinction between the type of academic study you described and the kind of rote learning and cramming required for tests like the SAT? They are quite dissimilar.

Sorry to hear that your classes at CMU involved just rote memorization and cramming, the two skills that standardized tests like the SAT seem to emphasize most. You apparently must not have availed yourself of course projects, discussion sections, or research experiences. I’m sorry to hear that the vaunted CMU educational experience has apparently gone so far downhill.

My experience was the same as all of my other classmates experience, and it is in the top 3 tech schools in the world.

I am describing what the best schools in the entire world do, and I have lots of knowledge from my classwork, and other classmates experience.

These are the best schools in the world.

And apparently someone else who responded had a similar experience at Stanford, another college that is among the best in the world.

You seem to have zero knowledge about how colleges work at even the literal best schools in the world.

Basically every top college in the world still has tests and final exams and studying and cramming. This is called the "normal college experience" at basically every college and top college in the world (and backed up by someone else in this thread from Stanford!).

What's going to happen is the transformation of people believing that you succeeding in SPITE of hardship is no big deal since you got a leg up.

I truly loathe those who "help" the disadvantaged by lowering standards for them.

it's hard to change a test enough that it's hard to prepare for but not so much that scores can still be meaningfully compared year to year. even if kids go straight to college, there's a two year window where they could reasonably take the test and the test is offered several times per year. I don't see how they could accomplish this without severely increasing the cost of the test.

> it's hard to change a test enough that it's hard to prepare for but not so much that scores can still be meaningfully compared year to year. even if kids go straight to college, there's a two year window where they could reasonably take the test and the test is offered several times per year. I don't see how they could accomplish this without severely increasing the cost of the test.

As an entrenched player, I'd love to know more about their operating costs. They have no need for marketing, have a constant demand and are essentially a monopoly (possibly part of duopoly) on the college standardized test market. If they had optimized supply chains and distribution (which I would expect to improve year over year) I would be curious as to where all the money is going.

Reply on news.ycombinator.com