Socioeconomic status and the relationship between the SAT and GPA (2009) [pdf]

1.) Intelligent people befriend other intelligent people.

2.) Intelligent people get higher paying jobs.

3.) Intelligent people live in wealthier areas.

4.) Intelligent people marry other intelligent people.

5.) Intelligent people birth intelligent children.

6.) Intelligent people use their intelligence to augment their kid's intelligence.

7.) Intelligent kids go to step 1.)

The emperor is walking around naked, and virtually all of academia is terrified to acknowledge it (namely step 5). Right there is the solution to 50 years of sociological fumbling to avoid saying "Intelligence has a strong genetic component, and we are not all born with equal mental capabilities"

Once academia can acknowledge this, then they can finally work on real solutions to close the disproportionate economic gaps it creates.

(Yes, I am aware of your anecdote that counters this. It's a broad generalization to counter the mainstream broad generalization.)

Relatedly, UT Austin found the SAT highly predictive and decided to reinstate the requirement: https://news.utexas.edu/2024/03/11/ut-austin-reinstates-stan...

> Those who opted in had a median SAT score of 1420, compared with a median of 1160 among those who did not.

> The higher standardized scores translated on average to better collegiate academic performance. Of 9,217 first-year students enrolled in 2023, those who opted in had an estimated average GPA of 0.86 grade points higher during their first fall semester, controlling for a wide range of factors, including high school class rank and GPA.

For me, I got much, much better SAT scores than my grades would suggest, because my grades were depressed significantly due to ADHD, (forgetfulness, never remembering when tests were going to happen, going off on tangents when studying) even though I actually knew the material. So the standardized tests were a chance to prove that I wasn’t a total dummy.

And you know, I think that’s kind of the point of a standardized test- grades definitely vary based on how involved your parents are, the particular teachers you have, whether you have adhd, and yes socioeconomic status- tests in principal are blind to all that stuff.

I can get almost all the questions correct on the SAT test without any time pressure but when I do it when the clock is ticking, I find myself unable to reason my way to the correct answer nowhere near as fast. As a result, I have to move on and skip 1/2 of the verbal and math questions because I don't have enough time. And I don't draw blank when under pressure so there must be something else going on.

I have tried to pretend that I'm an LLM and trained myself on hundreds of SAT questions on YouTUbe where people solve them. I've found that my reasoning speed under time pressure is a direct function of the amount data I've trained myself on.

Is there a quicker way to this or do people like me, with average IQ or Chat-GPT 3.5 if you will, have to train on thousands of math and verbal questions because we don't have the Chat-GPT4 like the gifted people where they ara able to be creative and don't need to rely only on past knowledge to solve novel problems?

The NYTimes had an good interactive story on this same topic this week:

https://www.nytimes.com/interactive/2024/03/09/upshot/affirm...

I would hypothesize that SES probably correlates very heavily with grades at the very bottom where you have things like abuse, abject poverty, children supporting their own parents, lack of access to quality nutrition, violence inside and outside the home, etc and even a genius would be swimming against the current. But middle class and above, I would expect the curve to flatten out. In my experience super wealthy people are more lucky than clever- it doesn’t take much brains to make more money when you’re already rich.

So, the goal here should be to identify smart kids in terrible situations, while not allowing rich kids who aren’t actually that talented to game the system. That’s the goal. And of course there will also be genuinely talented middle class kids and you don’t want to ignore them either.

i.e. socioeconomic status is correlated with SAT performance because socioeconomic status is also correlated with stronger basic academic skills.

Wouldn't socioeconomic status correlate just as strongly or even more with high school AP grades and extracurriculars?

It seems that these would require more discretionary time, as well as discretionary money, when comoared to the SAT.

So, I know the authors of this report personally. I sincerely have the highest respect for them, they do solid work; if I see something by them I take it seriously because I know I will disagree at my peril.

However, I also feel like findings like this get distorted and manipulated, and used to refute strawmen versions of arguments out there against use of standardized testing. Some of the recent arguments being used to reinstate standardized testing as required seem irresponsible to me, even though I don't necessarily see required standardized testing as undefensible per se (my own opinions about all of this are complex and don't fall neatly into either "side", so take this all as coming from someone who is frustrated with how things get oversimplified on both sides, not as advocating for a particular position).

Here's some things to keep in mind:

1. This is all about predicting freshman GPA, which itself is questionable as a criterion. If your sole question is "how will this individual do in our college their first year", it's reasonable. But if you are asking "how well would this individual master the skills necessary to succeed in X, Y, or Z role" it's entirely different. Freshman GPA predicts graduating GPA less well than you might think, and both of those are poor proxies for "real world" behavior. They're not unrelated to "real world behavior", but the relationship is low enough that this is the entire point of social justice advocates.

A lot of people would say neither SAT nor freshman GPA are relevant, that that's the point.

2. A correlation of 0.47 or 0.44 is something to pay attention to, but it's also far from perfect. If you want to remind yourself of how much noise is in that correlation look here:

http://www.analytictech.com/mb313/correl4.jpg

That's a ton of noise at the individual level, which is the level we care about when we're talking about admissions decisions.

The problem with a lot of this isn't with the SAT, it's that what happens is this 0.45 r inevitably gets turned into a blind metric that ignores all that real variation because it puts too much pressure on the school to take people who have all kinds of other evidence of competency, because it makes their numbers look bad.

Let me put it this way: if I was selling you a bathroom scale, and showed you that scatterplot of my scale's numbers and actual weight, would you buy it? You shouldn't, but we're making consequential life decisions based on that level of noise.

The Dartmouth report conveniently ignored that noise, when that noise is about 75% of the problem.

3. Relatedly, if you look at the individual level, things are vastly different when you start taking these kinds of population-level trends and using them to make individual decisions around a threshold. That is, it's very easy to say "0.46 correlation is pretty substantial, we should require SATs", but that 0.46 correlation is across the entire range. It includes people whose SAT scores are very very low. It's not only in the range of where decisions tend to be made, and to make use of that test as a metric for decision-making, you have to assume that what's in the college board's SES index is a perfect summary of what might be said for every applicant to your college. Maybe someone is upper class but their parents were both killed in a plane crash. Maybe they are middle class but come from an abusive family. Maybe they are applying as a much older candidate, or much younger candidate, and the meaning of the scores is really different.

In a sane world where we can have nice things, the admissions committee would look at this and make exceptions. This is the idea of having test scores optional. But when you make them required, and are required to post them, then there's pressure to take the highest scores regardless of all the other information.

4. About the other information: in Table 2, they show that HS GPA is actually less correlated with SES than SAT score. Other papers with similarly large amounts of data have shown that HS GPA is actually slightly more correlated with freshman GPA than SAT score. So is the SAT necessary? Probably not. We can argue about grade inflation etc., but the numbers are the numbers, and if it's working as an alternative that is less correlated with SES, reflects a longer sample of school behavior than just a few hours on one day, why wouldn't you prefer that?

5. Note their Figure 1b. They spend a lot of time talking about this model, and in some ways it's the focus of their paper, evaluating it against the alternative. But related to my first point, the model in Figure 1b says nothing about the extent to which a college, or anyone else for that matter, might want to select on some process represented by Figure 1b. What does it matter if test score predicts, say, some other test score composite, and both are influenced by SES?

In widely used intelligence tests, there were questions about things like 19th century European literature and classical music (I think they're still there, but I can't remember offhand; they might have been made optional now). On the one hand, yes, 19th century European literature and classical music is great, knowledge of it probably reflects some memory ability etc, but are questions about that really how you want to evaluate someone's skillset? Maybe it is, maybe it isn't, but I can tell you if you're not knowledgable about that you would not get credit on those questions.

In many ways, for many, the SAT and college freshman GPA are kind of similar. I don't feel that way, but this is the classic "book smarts" versus "real smarts" issue that always has come up since the beginning of humanity probably. Showing book smarts predicts other book smarts just isn't important in some paradigms.

Again, I'm not anti-standardized testing. I think it's useful. But I also think the way it's been used in the past, and continues to be used, is in fact broken. It's not even necessarily a problem with the tests, it's a problem with the way they get used. But to paraphrase a famous educational psychologist, if you have a thing that people tend to misuse, every single time, and there are good alternatives, maybe there is something about that thing that's a problem when you put it the hands of people, and the thing either shouldn't be used, or there should be some rules put in place with teeth to prevent it from being misused.