Report on the role of standardized test scores in undergraduate admissions [pdf]
home.dartmouth.edu
home.dartmouth.edu
>> Importantly, these test scores better position Admissions to identify high-achieving less-advantaged applicants and high-achieving applicants who attend high schools for which Dartmouth has less information to interpret the transcripts.
Precisely. SATs aren't about hurting the poor or disadvantaged. It's about giving them a chance.
Why is every parameter of admissions open to change despite tradition, except that one?
I can't imagine theres a dearth of physical space or qualified educators near any university. The fact that they are not growing to accommodate the demand has always boggled my mind...
Germany might be a better model where barebones education is easily accessible. They leave much more up to students: your homework might not be looked at and your grade depends solely on a final exam. No student centers or leafy campuses either.
The glut of un/under employed PHds suggest that the limited faculty is an artificial limit. Schools could easily hire more faculty and expand their undergrad programs if they wanted to.
They see their half percent acceptance rate as something that makes them special.
There's no shortage of people who can teach, there is only a shortage of a willingness to pay them to do so.
The problem is that the university departments are less willing to hire the faculty that they once did, and that largely comes down to university administrations having decided that paying for anything non-STEM (and non-sports, I guess) isn't worth it anymore.
The demand / revenue side might be another obstacle, especially if the easy money environment for school loans evaporates
Sounds like a community college.
That might be a concern for Dartmouth and the Ivys. But for US state schools, not so much. And those generally provide a very high value:cost product.
Also means that the "just increase the spots" already happened as well, there is no lack of spots today, just a lack of prestigious spots. But not everyone can get into prestigious positions, by definition prestigious has to be something special and not for everyone.
Sort of.
Sure, the school itself is huge, but often times that means that worthless majors[1] like Psychology and English are open, but more valuable majors are often limited.
---
1. Worthless in the sense of earning potential with a Bachelors
Dartmouth today has 6,700 students of all races, genders, religions.
Enrollment numbers can and do change, but it is much harder and slower to change something that is physically and resource constrained than it is to change policy.
And where that is a goal, there is clearly a point where you have to draw a line based on capacity, student:teacher ratios, etc.
Trade schools make a lot more sense for most. Get a good job and you can self study from a position of safety...
What did you have in mind?
However, if you go spend some time teaching at a community college, I don't think you'll walk away with the idea that somehow getting all those students pushed into a university would improve things for the community college students or for the ability to educate kids at the university.
Increasing enrollment might be beneficial, but past a certain point it is going to be detrimental to education.
It’s an objective truth that there are more elite students than there are positions in these schools. It’s also true that schools have absolutely exploded in administration size. They just haven’t hired more teachers.
Schools can’t double capcity overnight. But they currently aren’t making student body growth a goal. They prefer to keep it small and be extremely selective and exclusionary.
The flip side to this is that undergrad doesn’t matter. These elite schools don’t actually provide an elite education. They merely serve as a talent filter.
Maybe controversial but I really believe this lowers overall standards. You just can't find enough faculty to meet the high standards.
Look what happened when tech blew up - they hired like crazy. I'd argue the overall level of programming skill went down.
Sorry, but this is ridiculous. Many/most PhDs who want a faculty position will never be able to get one because the availability of jobs is so low. There's really no shortage of potential faculty members.
Standardized exams are the most fair and transparent.
Materials and tutors are available easily for free or cheaply on the internet nowadays.
perhaps the _most_ fair right now, but certainly not what I would consider fair.
consider a middle class family that has secure food, housing, and transportation vs lower class family where the child might potentially have less consistent sleep, food, or has to work to help pay bills.
one has more distractions than the other.
Move all children to government controlled facilities and do not let them see their parents until they are some age like 21?
Or maybe we should get rid of all criteria for education and just do huge once a year lottery. You get one chance to enter. As poor people can not afford more.
Then you gotta make it affordable for them.
You can still have elite schools for the absolute best (though I would question that certain self-styled 'elite schools' are really principally selecting for that and not e.g. wealth or family connections), but that doesn't mean that you cannot raise the bar on public and affordable education (including traditional schools and universities but also via other means, e.g. self-teaching via online resources). These are two competing goals and often people only focus on the one to the exclusion of the other, but they are both important.
This hasn’t been true for years, decades even
The only way to pay for a large and generous welfare state is to educate the most capable to the highest level of ability. The tech industry was built by highly ambitious, competitive, risk-taking individuals.
"California collected almost $1 billion in personal income tax revenue from Palo Alto’s 94301 zip code in 2016. That was the most of any zip code in the state, according to a new LA Times analysis, which found that California’s top 1 percent of filers paid nearly 46 percent of income tax for that same year. "
https://www.kqed.org/forum/2010101867426/californias-top-1-p....
Two, you don't label it as fair, and you recognize the disadvantages that exist and start talking about solutions.
I also don't see the problem with students from poor families simply going to community college for a couple years then transferring to a state school. It's far cheaper and the curriculum is largely the same for most undergraduate courses.
The major problem that progressive don't want to recognize is the role of culture and family structure, e.g. lower class Blacks and Whites who view education negatively, and who tend to be raised by single mothers. Asians generally don't have this problem and have the highest SAT scores even when you control for family income.
However they do offer a "Landscape" which provides similar details and many admissions programs use that to do something like what you're describing.
But that certainly isn't scalable. I'd argue the SAT is one of the most fair things we have at scale.
It's true that the SAT used to be biased (for example the word problems would use the word regatta which a wealthy east coast teen would know but a poor west coast teen would not) but from what I've seen they've done a good job removing that bias (as shown by this paper).
Admissions officers could then watch the recording, or rely on the automated scoring that the AI platform spits out.
Not everyone performs well on interviews, and I imagine performance would vary even more on interviews where you know your "interviewer" is an AI. But it could be really interesting, and a way to achieve the type of personalization and scale that you allude to.
For example in 2022, 63% of black children are born in single parent homes vs 16% of asian children. Having two parents around is an advantage in the doubling of resources.
What policy actions would you recommend to change what outcome?
https://datacenter.aecf.org/data/tables/107-children-in-sing...
>> Third, under test-optional policies, some less-advantaged students withhold test scores even in cases where providing the test score would be a significant positive signal to Admissions. Importantly, Dartmouth Admissions uses SAT scores within context; a score of 1400 for an applicant from a high school in a lower-income community with lower school-wide test scores is a more significant achievement than a score of 1400 for an applicant from a high school in a higher-income community with higher school-wide test scores. Admissions uses numerous detailed measures of outcomes at the high school and neighborhood levels to account for these known disadvantages. As one example, Admissions computes a measure of how each applicant performs on standardized tests relative to the aggregate score of all test-takers in their high school, using data available from the College Board.
And this is how you again discriminate based on race in a post-Affirmative Action world. The weight they assign to each context is completely arbitrary and tuned to achieve whatever objectives they have.
They're explicitly tying academic performance in relation to one's socioeconomic background. I find their claim of "doing more with less being more impressive than doing more with more" reasonable.
The only way to avoid any sort of judgmental/arbitrary calls is to blindly apply academic standards (the way UT Austin does top 4% of your HS class) and that comes with its own problems.
If you have two families that are equally poor, but where one family cares a lot about education and sends their kid to the harder school (perhaps by living in the cheapest apartment in the neighborhood that is eligible for that school), they will be disadvantaged compared to the other family, who send their kids to the less rigorous school.
To the extent that there are racial/cultural correlations that result in poor Asian kids going to schools that have higher average SATs than poor kids of other races, an ostensibly race-neutral policy like this will mean Asian students could (as a group) continue to face disadvantages in the admissions process.
In a post-SFFA vs Harvard world, anecdotes and thought experiments carry very little water. We all know - and the universities are very explicit about this - that socio-economic indicators are the new affirmative action. It's clearly the only practical way to effect the sort of diversity missions that many Universities seek to establish (since by their own published work, it correlates with strong community-wide academic and scholarly outcomes0. At this point, since we all know what's going on, the only question to ask is, "Why didn't you try to game the system since you know how it works?"
I'm not passing judgment on whether this is a good or a bad thing, whether it's honest or dishonest, or whether it's fair or not. It's just reality. SFFA v Harvard is a _horrible_ decision based on absurd jurisprudence and a total divorce from the truths of the world we live in. I commend everyone from admissions boards to clever families who learn how to play the new rules of the game better than those who backed the jurisprudence in the first place.
Yes, I think they would use this as a facially-neutral way to achieve a race-motivated goal.
> We all know - and the universities are very explicit about this - that socio-economic indicators are the new affirmative action. It's clearly the only practical way to effect the sort of diversity missions that many Universities seek to establish
Are you saying they want to have racial diversity or SES diversity?
> SFFA v Harvard is a _horrible_ decision based on absurd jurisprudence and a total divorce from the truths of the world we live in. I commend everyone from admissions boards to clever families who learn how to play the new rules of the game better than those who backed the jurisprudence in the first place.
Can I ask what you found so objectionable? I understand that many people disagree with the outcome, but from a legal perspective (IAAL), I haven't heard much criticism.
Because of structural racism and similar phenomena, they're not fully separable. It's not obvious to me they're even separable in a meaningful way (meaning that if you optimize for one, you simply won't get the same result as if you were to optimize for the other).
> Can I ask what you found so objectionable? I understand that many people disagree with the outcome, but from a legal perspective (IAAL), I haven't heard much criticism.
(IANAL, and it's been quite a while since I spent cycles on this so my recollection may be fuzzy) Fundamentally, I'm very uncomfortable with the dissonance in the concurring/dissenting opinions of Thomas/Sotomayor, respectively. My understanding was that they basically argue that the proposed standards by SFFA's lawyers are effectively unworkable, yet at the same time, there's very little to indicate that many - if any - universities were actually already out-of-compliance. In fact, in the aftermath of the case it seems as if most universities simply publicly doubled down on their existing diversity policies and missions.
So what's the point here? Why did SCOTUS accept to hear this? It seems like it was just taken up so that Roberts could fire up culture wars with the commentary he made in the majority opinion he authored.
It's hard to find virtue in something that claims to do a lot but in practice seems to do virtually nothing. So maybe it's incorrect to state that this is based on "absurd jurisprudence," but I can't find anything redeeming or positive to state about the opinion or the process that took us here.
They accepted the case because they wanted to end race-based affirmative action.
In terms of the dissents, they were completely out of step with the Court’s affirmative action jurisprudence. Going all the way back to Bakke (1978), the Court has never approved of any rationale for race-based affirmative action other than the ‘diversity rationale’. This states that colleges may use affirmative action in order to improve the educational experience by admitting more students of different racial backgrounds. The dissents employed fiery rhetoric but were not shooting for the diversity rationale. As a result, their arguments were destined to fail.
They typically go out-of-state, but the competition for the next 4% is insane.
> whatever objectives
They're very clear about their objectives too - they want more of their student body to come from a background of lower socioeconomic status.
The fact is if they're in a public school in an underserved area, they are most likely going to a disadvantaged public school, and face strong headwinds trying to achieve the kinds of scores kids in affluent neighborhoods get.
Note also that multiple factors are being looked at.
Generally when such systems are implemented the levels are not binary, e.g. housing project vs affluent suburb. My point was that for some bimodal neighborhoods using the neighborhood income average is not effective. Is there a national, annually updated list of neighborhood underserved-ness list?
This came up in Boston Public Schools. Admission to Boston Latin took into account the portion of a student body eligible for free lunch. It became mathematically impossible to be admitted to Latin if you weren't coming from a low income school.
I don't believe this is true, but if you have citations I'd be happy to hear them.
For instance, their accountability data [1] shows that, in 2023, out of 409 kids taking their ELA MCAS test, 92 were categorized as "low income."
I find this hard to square with your statement (all students came from low-income schools but 88% are not low income?).
1. https://profiles.doe.mass.edu/accountability/report/district...
A student attending a school without bonus points would need to score 97 points. Class rank is a factor which effectively imposes quotas on schools regardless of students grades and or test scores. Back around 2016 there were a handful of schools where students needed to score in excess of 100 points to get into Latin.
Last year 408 high income students received bonus points while 29 low income students did not. The admission rates for these groups were 85% and 55% respectively. [1]
[0] https://archive.is/20220203094548/https://www.bostonglobe.co... [1] https://archive.is/20231107020144/https://www.bostonglobe.co...
No they aren’t. They’re selling a product to the highest bidder but happen to work in an industry where social and cultural expectations don’t allow them to sell exclusively to highest bidders. Adjusting to those expectations is ever changing as we’re seeing here.
It’s an unfortunate fact that high bidder’s children roughly correlate with high potential (somewhat impolite to point out), so colleges these days can still point to success stories and pretend they did something.
This would be far more disastrous if spots at colleges were scarce, but they overwhelmingly are not. There are over 15 million undergraduate students in the United States, and we are undergoing a demographic collapse (https://www.populationpyramid.net/united-states-of-america/2...). In 2 years, the children of the baby bust (2008-2009) will start applying and enrolling in college, and because there are fewer of them, they will have an easier time getting into the college of their choice.
>95% of us do not attend Ivy League schools. Removing any and all affirmative action would help, at best, five thousand students? This is a manufactured issue. No one is denied a college education because of affirmative action. Any time spent arguing against it is wasted.
This would allow students to have a lower-stress life and have an advantage at college admissions — even if they actually learned less academically than they would have at the rigorous high school.
Of course, this is a dynamic system, and if enough high-scoring kids migrate to less-rigorous schools then the advantage would be diluted. But at the same time, kids who previously would have gone to that less-rigorous school will now be measured against a different standard, since their school would no longer be one that has a very low average SAT. Basically, it's complicated, and there are weird incentives at play.
I don't know. Look at parent's address, correlate with census and income, look at SAT, and be done with it. Solved lol.
the schools are in most cases public, in which case there is a mandate for many of these statistics. public money pays, so show us what you're getting for it. this is especially the case given poor and underprivileged students, who have been the focus of many studies (and some targeted funding efforts).
the middle school to HS to college pipeline also isn't new at all, and has been tracked for decades.
whats really more shocking to me is not that it's happening, but that more people aren't abusing it aggressively.
(I don't know how true this "key idea" actually is. I can imagine ways in which poverty might disadvantage people that would persist through their time at university. But it seems plausible prima facie.)
Since I encountered a similar approach just three years ago in my son's High School application in the Chicago Public School System I can say this leaves a lot to be desired non low-info,e families. Here's a high-level summary of how it works:
1. Chicago is neighborhoods are assigned one of (if memory serves) four levels, which represent socioeconomic levels.
2. With the thought that the economic level is proportional to school quality (this is more true in most places in the US than other countries, since a big chunk of school budget comes from property taxes, so neighborhoods with low taxes get school with lower budgets) scores from the entrance exams for kids from different levels are boosted by different multipliers.
Now, of course, (2) is a travesty of the US but that's another long discussion. The problem with (1) is that the neighborhood to level assignments are not revised each year. So you can be affluent in a district that has gentrified in the past 2-5 years and your kid will get the low-income boost unfairly.
Additionally some neighborhoods are highly bimodal, eg take Lakeview. This is considered a low-income neighborhood but there are many high-income high rises and single family homes within its borders. Alternately, there are many rental condos in our neighborhood which is the highest income level, so these middle income kids get slammed.
Still another problem is families purposefully gaming the system. We had friends who changed their rental condo during their child's 8th year so they can get a lower-info,e address sin their application. This is widely known and done.
Perhaps a better approach would be to use the family's income rather than communities.
Edit: I had more time to research and found the following, in case somebody is interested:
* What I referred to as the four levels are actually called tiers
* The tiers are actually updated every year, so I was wrong above. Still, the community average criticism stands. See the discussion about the update that was done last Feb here: https://chicagoschooloptions.com/forums/topic/when-are-cps-t...
* You can see the distribution of the tiers from a few years ago on this site: http://cpstiers.opencityapps.org, owners say it's no longer maintained. It seems currently there's no map showing all tiers, you can check only a single address at https://schoolinfo.cps.edu/schoollocator/index.html
(Yes, it takes time and commitment but it's mostly not the case that paying for test prep courses automatically translates into higher scores.)
And yes, money helps a lot to brute force what material works for a kid and get motivated teachers. It's not a magic bullet, but it's a lot better than no bullet.
Education is free and freely available.
"Maximize your score with free Official Digital SAT® Prep"
You should probably get that checked out.
> It’s worth acknowledging a crucial part of this story. Dartmouth admits disadvantaged students who have scores that are lower on average than those of privileged students. The college doesn’t apologize for that. Students from poor neighborhoods or troubled high schools have effectively been running with wind in their face. They are not competing fairly with affluent teenagers.
> I also asked whether she was worried that conservative critics of affirmative action might use test scores to accuse Dartmouth of violating the recent Supreme Court ruling barring race-conscious admissions. She was not. Dartmouth can legally admit a diverse class while using test scores as one part of its holistic admissions process, she said.
https://www.nytimes.com/2024/02/05/briefing/dartmouth-sat.ht...
I am conservative and I have no problem with poor students from disadvantaged backgrounds having their scores weighed differently.
I absolutely do have a problem with race-based admissions. Most conservatives have a problem with race-based admissions because it is not fair to poor students.
I remember a girl from my college (elite institution on west coast) that would go on and on about how disadvantaged she was because she was 'black'. Firstly, she's like a quarter black and looks completely white. Secondly, and more importantly, she's filthy rich. I mean... like multiple international trips a year growing up. Super picky about fashion, etc. Both parents had PhDs. Nevertheless, she qualified for all the diversity offerings the school had, including scholarships, special programs, etc.
Meanwhile, we had kids whose parents never attended college and grew up in the middle of bumfuck nowhere with very disadvantaged background, who were given nothing because of their race and sex.
Now, I'm certainly not part of the latter group (grew up middle class and am neither white nor black or any 'disadvantaged' minority) and frankly don't have a whole lot to lose in this debate, but seeing this with my own eyes has convinced me that we need to look at actual disadvantage, not just race. There are A LOT of high-achieving black, latinos, <insert minority here>. That's great! More power to them. I have no qualms with rich people.
I do have qualms when they pretend to be disadvantaged to get an advantage over obviously more disadvantaged others. This is annoying when Donald Trump does it. It's annoying when Elon Musk does it. And it's annoying if someone who's not white does it too.
The idea that simply because you're of a particular race, you're automatically disadvantaged is retarded.
I do hate to see poor and working class white male children pretty much hung out to dry in the eyes of progressives these days.
I completely agree that this used to be the case. I might even argue that it is still true to an extent. I will also agree that we should still address it. My problem is when addressing it involves discriminating against asian Americans like me. The people who want to discriminate against asian Americans are just using "historical disadvantage" as an excuse, their actions are political/ideological.
Please don't use slurs here. There's no need for that.
Is it really too much to ask people to come up with another word to use that hasn't been hurled at innocent people as an insult?
B. People who are of a certain race that get a 29 on their ACT are admitted over people of another race that get a 32.
It sounds like Dartmouth is doing A. The recent court cases had to do with schools that were doing B. If a college requires SAT/ACT tests and continues doing B, then it is pretty easy to see they are doing so based on objective data.
I believe those court cases are one of the reasons schools are moving to test optional--not because the SAT/ACT isn't useful to predict success at college, but because they see it as a liability in their acceptance process.
They may have a higher bar for "privileged" students, i.e. Asian/white, but the bar is far lower for the truly Privileged legacies/donors.
The point I was trying to make was that it appears many colleges may have an incentive to NOT require ACT SAT tests if they want to continue to use race or proxy for race and avoid what the report from Dartmouth is suggesting. Dartmouth looks like they are considering ACT/SAT along with the students background / opportunities, but specifically NOT treating people differently based on the color of their skin.
Dartmouth is assuming that they will get racial diversity just by getting better at selecting the best students instead of trying to penalize or reward people for the color of their skin.
There are other non-academic tests like CogAT, Wechsler, etc. that may be even better since they evaluate abstract reasoning and scale from young to old without significant modification. Those are used by gifted programs and organizations like Mensa. It’s possible that these would be adequate replacements for college admissions as well, but you may get a bunch of mildly on-spectrum kids who have never had to work hard for academic success (from my personal experience).
True, though if socioeconomic status were not correlated with race, then there would be much less of a reason to want to select based on race in the first place.
> There are other non-academic tests like CogAT, Wechsler, etc. that may be even better
They probably would, for a while. Anything different from the SAT/ACT but correlated with some sort of capability is going to do better when the problem is specialized preparation for existing tests. And that was precisely the problem that caused colleges to stop looking at SAT/ACT. It wasn't because the tests are bad. It was the usual "you make what you measure".
Switching tests isn't free, though. You lose all the historical data that show the association between a test and the specific outcome you care about (eg graduation rates at this institution). You have to go through a bootstrapping period, and by the time you're done, it's possible that people are now targeting the new test and it starts losing its utility too.
I think a lot of these types will probably also show up among the people who do extremely well on the SAT and ACT.
More generally, a 1600 is not some kind of "get into college free" card.
I did admissions for a very competitive university program many years ago and we primarily used test scores to establish a baseline. One reason was to make sure we only considered people who could handle the program's rigour, but equally valid was that the four of us couldn't possibly review thousands of applications so we needed an automatic filter. But after that filter we didn't pay attention to test scores except as a tiebreaker when filling the last few spots.
However, it sounds like their system works around this problem by weighting the test score based on the average score at the school. So they can still down-weight the application of a poor Asian student from a more challenging school, compared to a different minority student who goes to a school with worse SAT performance. If Dartmouth's percentage of Asian students doesn't tick upward materially in coming years, I would assume this is what they are doing.
To steelman the argument for this process, it's effectively boosting the scores of kids whose parents didn't care as much about education, or who had no avenue into a challenging school. On the other hand, this process dings students from families that prioritize education, who work really hard. Those kids would probably succeed at Dartmouth, and beyond.
I'm beginning to think a lot of the people commenting on these things have this weird concept of these schools admitting poor kids from the sticks and asian kids from Chinatowns with no resume items except good grades and test scores....it's true they have much lower standards for them, but they still need to have a decently "pointy" resume to get in.
The only way Dartmouth can fix it's "equity" problems is by shutting down for good.
FYI the middle 50 is 1480 to 1560 for the class of 2028: https://www.crimsoneducation.org/us/blog/dartmouth-acceptanc...
I know a handful of people that got into Dartmouth and equivalent Ivies, none had an SAT score that low and this is from middle class high schools in the surburban south, not even the rich public schools, selective magnet schools, or prep schools.
I remember a time when CUNY schools would accept nearly everyone but had more Nobel Laureates than all of China even today. Perhaps this is a better model.
TIL. I only knew the other definition, "the quality of being religious or reverent".
One of the most interesting studies, not specifically focused on "success" / GPA, but on "life-satisfaction" comes from a longitudinal study "Project Talent"[1]. The key correlation to "life-satisfaction"/happiness is how many questions a person _attempts_ to answer on an otherwise un-finishable exam.
> IMO the thing that tests do not sample is persistance
Some companies used to do this. Around 1900, if you went to work for a major US steel company, you started out shoveling coal. Even if you were destined for management. Japan Air Lines pilot training for new pilots involves a stage where they do grunt jobs in maintenance such as checking incoming part serial numbers.
> No, it's not hard to test for.
> ...the testing for that is insane.
I imagine if you told people "we are not testing for correctness, just number of problems solved" then you would skew this test.
However when the tests were administered in the longitudinal study, the researchers didn't know which metrics would be important so they just put a lot of problems on it.
Over four years, I added 350 SAT (1200-1550) points and 5 ACT points (30-35), but it seems pretty likely to me that it wasn't the direct work ("persistence") that made the difference as much as the generic four more years of schooling and general maturity.
If one sets a goal to achieve & takes many tries at it, it is similar but not the same as "heres a bunch of questions, answer as many as possible and some correlations in the answers might be more important than others".
I think if you want to predict success there are two key factors: 1. Test takers must have weak feedback on their results 2. Test takers must be told that there is not uniformity in the importance of specific questions & some may be worth more than others. The test must be long enough that no one is willing to sit for the entire duration, and test takers should not be given an end-time or instruction on how to take the test.
Why? #1 mimics the weak feedback in real-life scenarios, if someone doesn't find you likeable for some unknown reason ... you'll probably see less of them.
#2 gets at you may never really know whats key to your success & hopefully by trying you get "lucky".
Sure, taking it a few times meant I knew what it felt like to take it. I knew what the books were like and how the proctors would act. I knew what could go wrong and what kinds of distractions I should be ready for.
That's why I think I only actually got better at the same rate that I would have if I weren't taking the tests.
A carefully crafted test-taking strategy is an optimization that seeks to maximize the score. Scoring on the aforementioned test basically wasn't correlated with future outcomes in a significant way, ie it could contribute but isn't a major, main or significant factor in determining how happy or successful people who took it came out.
Retaking tests has these problems, in that people adapt to optimize the result. For example if you told people the result of this test before taking it, I would say you would end up with a lot more people "answering" in the fastest way possible all of the questions. This would destroy the effect measured, in the same way that telling people tests will be ranked by scores erodes the metric.
"Retaking" misses the implementation of this test, it is more akin to "I'm going to keep giving you problems as long as you are willing to sit here" kind of thing.
Additionally this is the main correlation of the study. I doubt if researchers had in mind that the number of questions answered would be significant. If you told students "we are testing for number of questions answered" obv would distort the results.
But also a good SAT for (those not naturally brilliant = most of us) also a positive indication of the student work ethic (to study for the test).
The problem I believe with the disadvantged students argument, which I believe to be a very real problem - is that lowering the standards wrt the test scores - is just a bad solution to a real problem.
Poor students do sit at a disadvantage - but the fix is much more difficult and complicated, and involves that child's entire life experience until they become 17 or 18 y/old. No one has a handle on how to fix it.
I get the feeling smart + hard work will come back into fashion when it comes to academic opportunities.
Obviously this is a proxy for socioeconomic status, and people higher on that ladder will have better support structures to do better in school.
This is begging the question as socioeconomic status is the primary indicator of educational attainment in our country today.
They aren't measuring potential, but the persons ability and advantages. Just as buying more TVs won't improve your students performance.
> Importantly, the relationship between first-year college GPA and SAT/ACT scores is likewise quite similar across neighborhood income and other demographic subgroups at Dartmouth. By contrast, Chetty, Deming, and Friedman (2023) show that certain non-test score inputs in the admissions process, such as guidance counselor recommendations, do not predict college performance even though they do advantage more-advantaged applicants at IvyPlus institutions, increasing their admissions chance
The question that hard to answer is did those lower performance groups still learn, and what was the delta on where they started.
Private tutoring, existing life skills, access to money to free up studying time etc...
A student that has to work through school or has to help with the career of a family member has to sometimes sacrifice GPA for life's reality.
Obviously GPA is important for social ladder climbing. But this is part of the reason social mobility is the US is far lower than many other countries.
A B or C+ student is still learning and allowing for a path for those in the lowe 'casts' is how we grow economically.
This isn't a zero sum game. Eugenics based policies have hurt our competitiveness.
This is also why the SAT was adopted by Harvard in 1934. Prior to that, most students attending Harvard came from private New England boarding schools that could only be afforded by the upper class. They saw an opportunity to diversify the student body by testing the general public and admitting those that scored highest. As an added benefit, the SAT was much easier to grade than previous essay-based admissions standards, and so it reduced cost for the University.
So their efforts didn't work.
Add that to the zip code correlation that is the strongest indicator of SAT scores and it does matter.
Students without parents that are college graduates are also at a disadvantage no matter what their potential is.
It is one of the barriers to social mobility.
It is possible both for SAT scores to predict a person's potential to contribute in cognitively demanding fields AND for there to be a strong correlation between SAT score and parental income/zip code/etc. In fact, this is true even of a theoretically perfect measure that can somehow peer into the future of every prospective student were they to be admitted.
Is there some reason you expect that the children of people who did not go to college will have the same level of potential as those born to college graduates? Would this hold true of the children of professors, or Nobel winners? Is there any place for 'nature' in your worldview, or is it purely 'nurture'?
If I flip your numbers around, it actually sounds pretty good: a full quarter of the alumni come from the bottom 80% of zip codes!
But note that teams built from cognitively diverse members are far more productive while homogeneous teams dramatically underperform.
https://hbr.org/2017/03/teams-solve-problems-faster-when-the...
IIRC most colleges correlate incoming and outgoing grades for specific high schools - so that a 3.0 average at a highly competitive high school (think Stuyvesant or Thomas Jefferson) is a better indicator of success than a 4.0 at a poor performing school - the SAT is very helpful to add context to a GPA for a students applying from schools where colleges don’t already have a sense for the HS program’s rigor.
I came to appreciate my high school education when I attended a highly selective university and found my grades going up from high school without actually doing any more work while seeing class valedictorians (I barely scraped by with a cum laude) from other high schools really struggling.
> First offered in 1926 by the College Board... Early proponents of the SAT argued that the admissions exam made higher education more meritocratic. Admissions officers at elite institutions like Harvard believed the test would identify talented applicants at less academically strong high schools and accelerate their journeys into higher education. [1]
Now it's a starting level filter. You're not getting in because of top scores, but you will get rejected because of bad scores absent a massive alternative hook. People that get in have top scores and other superhuman traits.
That's probably a good thing imo. It indicates that our overall expectations for our elites is on the rise, which translates to better outcomes on net down the line when they actually get fast tracked into positions of power. Those of us who are smart and lazy similarly benefit because many more of us are able to find less taxing jobs plying our expertise at the right hands of these elites for very comfortable wages, rather than being forced into the millstone of the C suite ourselves.
In today’s educational environment in America, students are rewarded heavily for belonging to an upper class, they are made to donate volunteer hours, join a sport team or a social club, participate in extracurricular activities, do homework (hopefully in a peaceful home setting with helpful parents and/or tutors), and then master the niche skill of pen and paper test taking. The SATs at the very least allow some students to skip all but last one, and have some chance of getting a decent education.
Ideally the systemic barriers to proper education should be dismantled. Remove this volunteer hour, move sports and social activities outside of the school system, remove homework, provide tutors to everyone that needs one, stop relying on pen and paper tests as the primary method of evaluation, etc. With all these systemic barriers removed, you can be sure that the SAT will have a diminished value.
Besides, SAT and ACT are really simple tests. If one gets 1600 for SAT, it does not mean this person will excel in academics, but if one gets 800 for SAT, well, this person statistically is not smart enough to achieve academic excellence. Of course, this person can be smart in many other ways.
That "all" is doing a heck of a lot of work, to the point of being indistinguishable from satire!
A lot of SAT success happens because of families who can afford to train and hire tutors specifically for preparation.
We know because our kids were two of those students. We took them out of public school into private last year. The first six months were hell as they had to catch up from their deficits and learn how to study for real. 18 months later they are getting real A’s and B’s (and a few C’s) that are real grades.
We need standardized tests because more and more school districts are going the social promotion route
The fix is to get proper leadership in the schools. Our Superintendent started his career as a gym teacher, and it shows. At a school board meeting when parents criticized the low standardized test scores, he crowed that they don’t matter because colleges stopped looking at them.
In our State, NJ, in theory each County has a County Superintendent to over see schools, and the State Dept of Education to oversee them. In reality, the County and State bureaucracy does little to nothing to provide actual oversight, and schools become private fiefdoms. The end result is school quality varies wildly, and housing prices are very strongly correlated to school quality. In our crappy district, real estate agents complain it is very hard to sell housing to families with children or who will soon have them because the schools are so bad. This creates a downward spiral for the school as fewer and fewer kids are enrolled.
the war against them was because people didn't like the implications that poor Asian immigrants were able to outscore other demographics regardless of income, which goes against modern academia's favored nurture over nature mindset and destroys the basis for trillions in social programs. Standardized tests effectively threaten the entire house of cards, even more than the replication crisis showing that most of the "research" used to justify modern social programs was fake in the first place
Ironically, the problem with them now is that they're now used to exclude white people.
back then being against eugenics meant you were a crazy religious nutcase who couldn't accept scientific fact about evolution and genetics
How many of those cultural and socioeconomic factors were almost directly related to racism though?
Yet Asians do well in California schools.
"The California Alien Land Law of 1913 (also known as the Webb–Haney Act) prohibited "aliens ineligible for citizenship" from owning agricultural land or possessing long-term leases over it, but permitted leases lasting up to three years.[1][2][3]....... The law was meant to discourage immigration from Asia, and to create an inhospitable climate for immigrants already living in California.[7][8][9]"
https://en.wikipedia.org/wiki/California_Alien_Land_Law_of_1...
Except legacy admissions and pay-to-play donations. After all, pedigree and wealth are merit (or proof thereof) - no need for pesky test scores in the face of such overwhelming evidence.
None of that changes what the poster you were replying to was getting at. The opposition to standardized testing was entirely due to the performance of _some_ minorities and entirely political.
This might be true in aggregate, but I’ve seen data that says otherwise (buddy owns a test prep school). The keys are that the student wants the test prep and that the student has room to grow (i.e., not already near the top, where increases are capped).
Middle of the pack students can gain 50-150 points per skill (this is a lot).
> the implications that poor Asian immigrants were able to outscore other demographics regardless of income, which goes against modern academia's favored nurture over nature mindset
Is this implying that Asian immigrants do better because of genetics (nature)?
The non-agenda-based research on this topic is pretty clear that Asian immigrant success is mostly about a culture of schooling and learning in Asian communities. This includes things like tutoring, “cram schools”, and test prep.
Here is an example of research/views from outside the US:
https://www.otago.ac.nz/deepsouth/vol2no1/setsuo.html
Almost all academic success is built on a culture of learning of some sort (e.g., parental involvement in schooling, reading to kids, etc.). Note that this is true regardless of race. The higher quality (non-agenda-based) research in this area all focuses on how this culture of learning can be fostered, with special focus on low SES (low socio-economic status) families regardless of race.
> destroys the basis for trillions in social programs
> Standardized tests effectively threaten the entire house of cards
Programs like Head Start in the US have measurable positive impact:
https://www.brookings.edu/articles/the-long-term-impact-of-t...
Fwiw, I actually agree that a lot of research in education is non-replicable, agenda-based bullshit, but your comments here go against both the better research and fairly substantial anecdotal evidence (my buddy’s test prep program).
Something can have an unjustified basis, but have a measurable positive impact. Just because the unjustified basis of race was correlated with income. But using the justified basis of income and wealth will not only lead to even more measurable positive impact, it will also be fair.
Yes, that is very likely true. I would go as far as to say that the "nurture" side of this coin is to a large extent an indirect result of nature as well.
I’m not sure if you’re Asian or have lived in Asia, but…
I have lived and worked in East Asia for 9 years, and I have worked with Asians (both Asian-Americans and Asian immigrants) for decades more, and I will just say that this strikes me as patently untrue.
East Asians (from East Asia) are often better educated as a group, but what they do with that knowledge is often underwhelming.
If a study were to control for things that correlate with intelligence like “was able to work with / game a system that led to immigration”, then I might agree that certain immigrants might have IQ that skews slightly higher. That said, the predictive variable there would not be race, but the ability to understand and work successfully within a complex system.
I have many beefs with SAT and standardized testing in general, but the link with IQ is not (anymore) one of them. IQ is a horrible and racist concept that should remain along with phrenology as a pathetic attempt to use science to explain racist beliefs. SAT used to be there. But we have to give them credit for having moved on.
In psychometrics (which is a scientific dead end IMO) IQ is indeed still popular, but the predicting factor of IQ is more often then not other operationally defined concepts which psychometrisians fight tooth and nail to justify is not circular logic, that IQ is not just predicting IQ.
If you think IQ is the best predictor of human behavior, you must not have heard about many more. If you grow up in an english speaking household, you are very likely to speak english fluently in your adulthood. You are much quicker to find Waldo if you’ve done so 10 times in the last 20 min, then on your first attempt (cognitive priming), if you smoke weed you take much longer to find Waldo. If you inhibit your serotonin reuptake in specific areas in your brain, you are much less likely to suffer severe depression. If you reward child with a praise you are much more likely to see this child repeat the behavior then if you punish the child with a shout.
These are all pretty powerful and—more importantly—useful predictors in behavioral sciences. IQs predictive power is very limited to a very narrow scope inside psychometrics, we will do just fine throwing it out. There are much better constructs out there (including the SAT).
This aligns with my experiences during my ADHD diagnostic process.
Perhaps IQ can be useful to some degree at population levels, but I am not even sure about that -- way outside my area of expertise. I do strongly believe IQ falls apart at an individual level. At least, past a certain extent. I imagine IQ might be a decent predictor of outcomes for an individual if his or her IQ is below one standard deviation of average i.e., (IQ <= 85). I'll even concede maybe up until the point of average. Past average, I question the utility, especially the further one's score increases.
Person A has an IQ of 125, and person B hasn't an IQ of 130. Realistically, how much of difference can we expect in life outcomes for these people? How much does each single point actually matter? I've never been able to find a good answer to this either.
Another issue I have is that IQ testing seems to be a snapshot of all of one's particular factors and circumstances at that given time. I liken IQ testing to something like hunger testing. I do not believe intelligence is generally a constant value nor is something like hunger. Both are qualitative by nature anyway.
I'm sure we could create a "Hunger Quotient" and use it to measure hunger the way we measure intelligence. However, wouldn't one's hunger vary depending on prior circumstances and factors like when the last time one ate, how much one ate, how many calories one has burned since his or her last meal, etc.?
I imagine IQ is no different. If I deprived myself of sleep for more than 24 to 48 hours straight and then took an IQ test, then I wouldn't be surprised if my score was significantly lower than taking it when well rested. There are probably damn near infinite amount of factors -- Do you have test anxiety? Do you have any chronic conditions? Any recent, major life stressors? Do you even care about doing well on this IQ test?
My point is that IQ can only tell you how "intelligent" you are at that given point in time when you took that exact test. That makes me question the utility of it even more.
I understand research tries to control for these types of factors, and that the tests seem to have great reliability when retaken. However, in my experiences, it does not seem like there is any attempt to control for variables in a clinical setting where these test are most often administered...
Finally, I would like to mention Lewis Terman's fun experiment which I believe he started in the 1920s where he followed a group of 1500 or so "gifted" children throughout their lives and for the rest of his life. Of the 1500 or so gifted children, while still mostly successfully, apparently none of them went on to become anything noteworthy.
However, two children that were not included in the study due to having too low of an IQ, William Shockley and Luis Alvarez, both went on to win a Nobel Peace Prize separately and in separate fields.
I'm not saying this proves anything, but I do find it rather funny.
How then do you explain IQ's strong correlation with positive life outcomes like work success, educational achievement, happiness, etc
https://www.scirp.org/journal/paperinformation?paperid=74943....
https://www.cambridge.org/core/journals/psychological-medici...
The educational achievement correlation has to be thrown aside completely because the SAT and GRE were largely an IQ test until the late 80s.
I'm not saying that IQ tests are completely useless. But clearly, the correlation isn't strong, and the whole hypothesis that it measures a g factor is not borne out by strong evidence.
The is a simplified way of putting it by I would roughly agree with this statement. However, I think you're drawing the wrong conclusion from the fact that "poor Asian immigrants were able to outscore other demographics". It is absolutely about nurture, but not in a way that the "academia" wants to emphasize. I think the main reason these poor kids achieve better is the support and, yes, the tremendous push they get from their families on topics related to education. I'd wager that having a two-parent family that places importance in education correlates most with academic performance.
"Using detailed admissions data from IvyPlus institutions, Chetty, Deming, and Friedman (2023) show that SAT and ACT scores also predict career success, including high levels of earnings and attendance at elite graduate schools, holding family income constant."
Or if you're more cynical, they failed to produce those results.
I know some really, really unintelligent people who got good grades in college. They just ate books.
I suggest these effects are because being a good student aka "eating books" is correlated with conscientiousness. They show up to lectures, prepare, and test well.
And conscientiousness is very highly correlated with lifetime achievement, AND fufillment [4]. So measuring conscientiousness, and signalling high conscientiousness is a really good idea.
IQ is great, but conscientiousness is how you get things done [5]
1. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9004755/
1.b (edited) https://www.washingtonpost.com/news/wonk/wp/2014/05/20/heres... might be better. I thought this was "common knowledge"!
2. https://mpreiner.medium.com/what-is-the-impact-of-your-high-...
3. https://www.ssa.gov/policy/docs/research-summaries/education...
That is...an extremely narrow study to use to make that broad an assertion. GPA of two classes from 2010 at a single business school in China, their starting salary, and then their salary in 2018?
The larger point is that, since you can study for tests (and GPA), that I think conscientiousness dominates.
https://www.washingtonpost.com/news/wonk/wp/2014/05/20/heres...
I would not give in to their assertions so easily if I were you.
I haven't read the entire study mostly because I don't care, but I think you're wrong in your statements.
GPA is well correlated to earning potential, as well as earnings in reality.
of course but how would these be measured
If that figure were about anything else, people would be screaming bloody murder about misleading figures and overly generalized interpretation.
I'm in favor of allowing for the use of test scores but they get abused and the language in this report is a good example of how this happens. Scores have these real but modest correlations with real world situations, but then get used as rulers of atomic precision without any context or recognition of their massive limitations.
It makes the authors of this report look either deceiving or ignorant of statistics or both.
I suppose this shouldn't be surprising. School does not train you to be a good office worker.
[1] https://www.linkedin.com/pulse/does-college-gpa-predict-job-...
Job performance is subjectively measured by your bosses which has political implications. It is inherently a terrible metric.
Who defines this "goal"? Each student has their own individual goals. I bet you if you surveyed students
"The goal of school is to create the ability to think creatively, rationally and critically and make you a better citizen of society through those processes."
Would rank all the way to the bottom and they all are terrible reasons to get into $200k of debt.
Many go to college without a second thought just because they think it's what they are supposed to do. Many go because they don't want to be a manual laborer and they want to be able to make more money. Many go to get drunk and party.
Good job, starting off strong with a real substantive criticism.
> I would imagine that most people attending college at least intend to learn something and level up some ability.
Your imagination is lacking and you have not spoken with many 18 year olds, the vast majority are not highly motivated learners. A large number of degrees teach skills that are not even valuable enough to pay back the amount of money that is put in. So they are leveling up in worthless shit. All these indebted graduates are clamoring to have the government pay for their bad decisions. None of them blame the institutions that charge these exorbitant amounts to supposedly become "cultured" and "creative".
The honest truth is they shouldn't be going to college. Because they went to school and didn't come out cultured or creative.
(I went to a state school. It was easy to structure your schedule with zero liberal arts. I almost did. To the extent I have regrets about my time in college, it's in not taking more liberal arts classes.)
I also went to a state school, I took the liberal arts classes, they have added nothing of value to my life. If you want to get cultured, read.
Sure, many were useless. But the few that were good were golden. Sorry you missed that.
> If you want to get cultured, read
This is necessary but insufficient. The discussion is essential. To be clear, I'm not saying you can't become cultured outside college. (Of course you can.) But that coming out of college uncultured is close to a waste for most graduates, i.e. those graduating with a degree that isn't immediately in high demand at a six-figure wage.
If a student isn't a highly-motivated learner, and their goal is maximising lifetime net earnings, they shouldn't be going to college.
I disagree. I think further education in valuable topics is valuable to people even if they are not motivated. I don't think getting into massive debt for this education is valuable to them. There is nothing inherent about college that requires it to be so expensive and simultaneously useless.
All the defenders of the current system always respond with the tired "college is not about teaching how to do a job, it's about teaching culture, creativity, and critical thinking" bullshit that is not backed by any data. If we go by actual results, college is about getting people into massive debt to fund college administrators.
What college and education should be about is teaching people things that can give them the ability to contribute to themselves and others.
Sure, but that doesn’t need to be college. Many European countries have colleges in name only that actually function as trade schools. An unmotivated learner should go to a cheap 2-year trade school and then start earning.
> college is about getting people into massive debt to fund college administrators
Completely agree. That said, we have standout colleges where the purpose is to educate our next generation of elites. That’s still important.
I think that means you were very fortunate. For me, 2 years at a community college was essential for getting my career in tech off the ground. Learning to write at the college level, being able to read/search/cite journals, and the practice of being able to synthesize a bunch of information into a cohesive document and/or presentation opened a ton of doors for me.
I get the impression some people figure this stuff out either through more demanding work in high school or from having smart parents who inculcate this knowledge into their kids. For me, I didn't have that.
We eventually broke up, and she went to college, changed her major 3 times, spent the whole time drunk at parties, and graduated with a degree with a relatively limited scope of jobs. She graduated 4 years ago, during which she's worked as a receptionist, a dog groomer, and a call center rep.
We shortly got back together as she was graduating college, and I encouraged her to apply for jobs for which her degree would come in handy. Every single time, she refused, because she "wasn't qualified enough" even though she had her Bachelors in what they were asking for.
I say all this to say that I think many kids are corralled into college without an actual end goal in mind. They go because it's what they're supposed to do. In my anecdotal experience, there isn't much thought about intending to learn or leveling up some ability.
Obviously, the worth of the metric goes down over time as first-year curricula differentiate from each other.
> But is GPA the best measure, especially first year?
No. For example, SAT score is a better single measure than GPA is. But you can't use that to check the validity of SAT scores.
Note I am not saying that people who don't do particularly well on SAT/ACT can't also succeed on college and beyond, I had a pretty average score myself and consider myself fairly successful. But all the people I know who had the highest scores on SAT's were also the people most motivated to study and work extremely hard whether it be based on family pressure or just an inherent drive to be better than everyone else, and even to this day these people I knew who had scores in like the top 95+% are for the most part the most successful people I know primarily just due to an inherent drive to succeed at all costs.
Of course this will also depend on what metric you use for success. Also of course there will be people who do extremely poorly in traditional education but become wildly successful.
Anecdotal evidence, part 1:
I grew up in the suburbs and went to public schools that, at the time, were good-not-great within the state. In that setting, I was a high achiever by any sensible metric. I was friends with a lot of the other "smart kids," and I definitely worked harder than some of them, but outside of school, I was also working on different things. My hobbies and extracurriculars weren't strictly academic, but they certainly set me up for academic success better than sports or video games would have.
Anecdotal evidence, part 2:
One of my classmates was "freak of nature" levels of gifted. I shared math classes with him for three years of high school, and to my knowledge he got one math test question wrong in that entire time period. He was also one of the school's best tennis players, and he made it into so many state concert bands/jazz bands/choir groups that their schedules overlapped and he couldn't do all of them. Last I checked, he was finishing up a PhD in neuroscience. But get this: he was our salutatorian, solely because someone else who did no sports and few clubs took more summer classes, and thus had the same GPA with more total credit hours.
Which means: you don't have the same stats for 3rd and 4th year students.
A pretty useful model for life is that it's a series of contests, and doing well at the previous contest gives you an advantage for the next couple contests, but only the next couple contests. By the time you get to mid-career, nobody really cares what your high school GPA was. However, because each contest determines which set of subsequent contests you'll face, performance early on can have outsize effects on eventual life outcomes. You typically won't be applying for CEO jobs if you worked retail your whole life, unless you lie your ass off and bullshit convincingly to executive recruiters.
It could just be that professors don't want to invest a lot of time designing a bevy of coursework. In the weed out classes they are all lecturing off a textbook, which provides for them slides to use and a general schedule of topics, as well as banks of questions they could pull from and permutate and reuse. Its a lot of time saved for sure. In the upper division, there is no textbook or anyone designing any course, they might literally have a sentence description of what the course is pitched about in the catalog then they take it from there. The professor has to come up with what to talk about from their field for an hour twice a week, and sitting there drafting up those slides is enough work if they don't have any from last year to use, considering they are usually also a full time researcher on top of this.
When I went back and was paying for tuition out of my own pocket as opposed to magic sky money falling into my lap from the US Dept of Education, things were a LOT different.
The options in my area after high school were kind of unattractive, though. I could work retail, or maybe go to trucker school, or try to find odd jobs helping people with computers. My current career path (veterinarian) didn't occur to me at that time.
Being a good test taker will 100% translate to doing well in college. Almost every course I took in college had over 80% of it's overall grade being tests, with a large number of them being closer to 90-95% for how much tests+final count.
So yeah, if you are good at taking tests and/or studying for them effectively... Yeah, you will do well in college. Especially the first year.
I'm out of the loop since my college days are behind me, and I don't have kids.
I had heard colleges were going to rely less on standardized testing (and many dropped the requirement during Covid when SAT tests didn't take place at all). I had also heard that there was controversy over reliance on standardized tests, because the wealthy can more easily afford expensive test prep, which initially seemed reasonable to me.
But then when I listed to the podcast I was stunned that the colleges seemed to think relying much more heavily on grades and extracurricular activities was somehow an _improvement_. I thought "grade inflation" was a big worry over the past couple of decades? I thought we all joked about today's nonsensical GPAs where kids get, whatever, 4.5s or 5.0s on a scale of 0-4. And it goes without saying that it's obvious to most of us that volunteering and participating in travel sports and debate teams and so on, is FAR more accessible to those with money, with transportation, with resources, with involved parents?
Standardized tests can't tell us how diligently kids do their homework, how well they do after encountering a setback. Maybe they favor kids who are smart but lazy. They can't tell us how a kid will do when they go from being a high school standout to a below average Harvard student.
But they sure as hell tell us a lot more than utterly subjective grading systems with no standardization whatsoever, and "intangibles" that tell us only how good a kid's family (or hired college prep expert) is at identifying activities that will get them into a selective University.
1. https://www.nytimes.com/2024/01/17/podcasts/the-daily/the-wa...
fwiw GPA and SAT scores are highly correlated anyway
Render unto Caesar the things that are Caesar's.
The SAT and the ACT are standardized tests, meaning that everyone in the country writes the same test on the same day, and the tests are designed to make the test results comparable between different sessions.
> state ran standardized tests
The US is the land of capitalism, so it delegates standardised testing to private for-profit corporations. Many European countries have a nation-wide or state-wide standardised testing system.
> Then there is this idea of the extra tutoring people are paying for outside of the school curriculum basically amounting to something akin to paying for the right to use steroids in sports.
I wouldn’t compare tutoring to steroids. In sports, some athletes can afford to spend more time in the gym and to get a better coach, while some can’t, and that might affect their test results.
It's not trivial to go from performance on a single test to performance throughout the year, which will include long-form writing, lab work, and other forms of learning.
60ish percent is a very particular number, because it's the highest weight that doesn't make it possible to pass the class by doing almost nothing except the exams. At 65 percent, you're giving students an easy way to game the system.
Some classes had a substantial lab component, but it was relatively easy to score well on labs. Labs were graded by TAs, who were consistently more lax than the professor. I don't think there's some coordinated effort by TAs to lower the bar; they were simply too busy with their own studies to provide a thorough criticism of every lab report they received. In most classes, labs were effectively a curve.
English courses were different, since those had essays instead of exams. However, I'd argue that it's feasible to write a paper "to the rubric" in the same sense that one can shape their studies "to the test." Given the audience HN caters to, I'd imagine that we're more interested in the science/math/engineering stuff anyway.
> You're delusional if you think that you're gonna get a 4.0 by half-assing course work
Nobody employing such a strategy was trying to get a 4.0 in the first place. In fact, setting the benchmark GPA for rigorous postsecondary education at 4.0 is itself delusional.
> The SAT covers fairly basic topics that nearly every student should know, that's why it's like an IQ test
Having taken two IQ tests administered by two different psychologists, I can confidently say no. The SAT does not resemble an actual IQ test. The SAT doesn't test short-term memory, and it only indirectly tests associative thinking.
That said, there's probably a correlation with IQ test scores, because smart people are smart, and so they perform well at tasks designed to test intelligence... just like how being good at taking tests in high school, means you'll probably be pretty good at it in college too.
https://slate.com/technology/2019/04/sat-prep-courses-do-the...
Not to mention, what about us test-disabled people?
I didn't know I had ADHD and its "friends" when I took the ACT, and I could have potentially benefited greatly from the reasonable accommodations the test allows for. However, it's not like every child is required to have some kind of psychological evaluation while in school.
I find all of this standardization in testing to be nothing more than Goodhart's Law in full swing.
Just what we need, more "Teaching to the Test" and route memorization in our education system...
I applied as an international student to a top tier liberal arts college equivalent to Dartmouth. I was able to make my case and was accepted there. My academic performance in both my first year and the rest of the four years was stellar.
Perhaps the SAT can be predictive of performance for those who grew up in the system that prepares you for the SAT (or ACT). For anyone else, this is however meaningless.
International students are a completely different story. As you say, your entire educational experience had been in another language. This is not the case for students in the US. All students here have taken many multiple choice tests in English by the time they get to the SAT. For example, there are state- and federal-mandated tests that students must take.
> Perhaps the SAT can be predictive of performance for those who grew up in the system that prepares you for the SAT (or ACT). For anyone else, this is however meaningless.
It is perhaps close to meaningless for an international student, but for a student who went to school in the US, it is not meaningless at all.
You probably did well at a US college because you came from a system that actually provided you with critical thinking and reasoning skills. Instead of a system, like the US, which is rife with route memorization and multiple choice tests.
I would imagine the opposite experience would be much more difficult (assuming what I remember from studying German is actually correct).
Apparently it's been discussed on HN before: https://news.ycombinator.com/item?id=37059857
Two-faced hypocrisy at its peak.
The university admission process takes a more comprehensive look at candidates than is practical during the hiring process in industry. In particular they pay close attention to proxies for general intelligence (SAT math and to a lesser extent reading) and trait conscientiousness (grades, extra-curricular, etc.). If an employer sees Dartmouth on your resume, they can make an educated guess about your IQ and your conscientiousness, both of which they should care about as an employer.
As other comments have mentioned, measuring these psychometric traits more directly has the effect of identifying top candidates in unprivileged circumstances. While diluting these psychometric scores with other measures has the effect of biasing towards affluent candidates who can afford to do things that are impressive on paper.
The paper's summary implies that they should make testing mandatory, because when they made testing optional, they couldn't see the students who they'd like to admit. But they kind of ridiculously don't comment on the fact that mandatory testing might not make any difference at all.
There must be reasons students don't send in scores when testing is optional. But just making the testing mandatory doesn't address those reasons at all. So there is no reason to conclude that suddenly applicants would start sending in scores at a higher rate. It's equally likely they would just stop applying.
I'm ADHD-C, but I always did well in school, got shit on by the ACT, went to college and did well up until my final year, but that was mainly due to some extenuating circumstances and not my drive or intelligence.
They accepted me (same year as the guy who wrote that article!) from a high school than only gave letter grades, computed no class rank or GPA, and offered no "AP" classes or SAT prep. Then again, back then, I don't remember colleges caring about extracurriculars either, though I might have forgotten.
Sometimes people say, that for ACTs, one needs a specific knowledge of how to take the test, which does not teach you anything, but I did not get the feeling.
All of the questions seemed quite reasonable, no tricks and not even a need for smart ideas (unlike let's say math Olympiad). Basically know the basic math rules, and you'll be fine.
The knowledge I've gained at a Czech Gymnasium with more traditional methods (i.e. no multiple choice) translated very well into getting a good score.
I was raised in the Southeast, USA. That's a big ask of us and our education system.
In my state, we have a few state administered tests that are taking throughout one's primary schooling years. Student take these exams at periodic intervals. For example, there is one for reading comprehension and assessment taken, iirc, at grade 3, 6, and 9 or something like that.
Well, 60% of students in my state right now who took the 3rd grade reading comprehension assessment have failed to demonstrate the ability to read at a 3rd grade level.
I have a hard time accepting these test [ACT/SAT] as some kind of arbiter or one's future when the education system is failing many students long before he or she ever gets to take the SAT/ACT.
The thing about these tests is that they are all perfectly reasonable. The main reason people have a test with them is they also tend to identify people who _probably_ should not be in college at all and that is not a message anyone wants to accept needs to be sent.
Meanwhile there is little discussion of why issues of equity in admissions are zero sum rather than leveraging multi-billion dollar endowments to expand access overall.
Some of those are amenable to short-term growth; others are not.
Endowments, like universities, are meant to be perpetual, so the amount of access they can provide is limited by the return on capital. Some people would perhaps like to see the principal spent down, which could provide dramatically more access in the short term, but then the money would be gone.
if dartmouth took sat scores into account in the admissions process by precisely the right amount, then within the set of people they actually admitted, sat scores would have no predictive value with respect to academic achievement, because the students who were admitted despite low sat scores would have been admitted due to some advantages detected by admissions that turned out to matter precisely as much as the higher sat scores would have
if, across the entire population, higher sat scores had no predictive value with respect to academic achievement, dartmouth could get this result by preferentially admitting students with lower sat scores. (for example, maybe you would need a high school gpa of 4.1 or higher to get in if your meaningless sat score were 1500, but this hypothetical bizarro dartmouth would let you in with a high-school gpa of 3.1 or higher if your meaningless sat score were only 1000; if the 3.1 students in high school continued being 3.1 students at dartmouth, it would make the meaningless sat scores highly predictive of academic achievement at dartmouth, among the people who actually got in.)
of course that would be an extremely unlikely thing for dartmouth to do, and they mention a lot of other evidence that sat scores predict academic achievement in general, not just among the students they admit
r-squared of 0.22 is still not a great predictor, though
if you want to read this report, it may be useful to open it in two separate browser tabs, scrolling the second one to the figures, which begin on p. 9
By sheer dumb luck, I am exactly the kind of intelligent schools and standardized tests measure for. I went to my first choice schools on a full academic scholarship for both undergrad and grad school.
And it's meaningless in the real world.
The most important connections I've made certainly weren't at school. The academics themselves weren't terribly useful in terms of preparing me for life outside the ivory tower.
Why are we trying to sell people on this dream?
It's not meaningless in the real world. You may not be satisfied with your level of success, but that's dependent on how you define success.
You won't find many dumb people among self made multi-millionaires. But being high IQ isn't enough to get you there. It also requires creativity, people skills, a network, luck, etc. We don't test for those things because the people running the show aren't interested in identifying people to compete with them, they are interested in identifying people who will work for them.
I'm also garbage at just doing what I'm told. Who's giving me the directions? Why should I trust them? What do they know?
You seem to be thinking of the people who did well in school because they worked hard. Perhaps they are excellent employees.
Those of us who never had to apply ourselves to anything and were told incessantly how much better we were than our peers make terrible employees in most atmospheres.
Living in America now I see a lot of parents spending thousands of dollars on SAT tutoring & preparation.
It seems a lot of things could be fixed just by being able to support and admit more students at the "good" schools? (also not sure why there is such a massive disparity in the quality of colleges here as well)
In Quebec, it's not present, instead you get in based on a Z-score renormalized based on how good your class was at standardized tests in CEGEP, which itself disproportionately looks at your standardized tests scores and those of your peers in high school
As far as the rest of Canada, highly competitive programs have to calculate the strength of a high school to normalize for grade inflation, especially in Ontario. It's a huge problem and makes it unfair for some students who don't go to the right high school.
But either way, it's a pretty good proxy for IQ.
I've seen years ago a highly cited paper showing relatively high correlations between SAT scores and IQs, but critically its data was from the late 1970s, when norms around test prep, and access to prep resources, were meaningfully different.
https://en.wikipedia.org/wiki/Raven%27s_Progressive_Matrices
Given that that test has a limited number of questions and the nature of the test it seems that anyone who is good at remembering lines could benefit from coaching. In fact they've come out with versions of that test which are different from the standard test because the standard test is too well known.
My perception is that there is a psychologist who administers the test and another psychologist who administers test prep, but a lot of people hang onto the idea that Alyssa Milano must have really gotten a perfect score on Raven since everyone successful must be really smart, the system is fair, etc.
They told me that if your kid has a 4.0 GPA (a perfect GPA), do not take the SAT/ACT ... because, without those standardized test scores - your kid is "perfect".
But unless your kid has a perfect 4.0 GPA, taking the SAT/ACT typically helps your child get into a better school than their GPA might indicate.
YMMV
You won’t get into a high tier private university (Ivy or not) without SAT/ACT scores.
99% of kids don’t have a perfect 4.0 GPA.
But the point is, if you do happen to have a kid who does - there’s more harm than good that will result from taking SAT/ACT.
For all others, it’s highly recommend to take.
My point is that, 4.0 GPA or not, if you don’t take the SAT/ACT you will be at a disadvantage when applying to elite, private universities. For the very simple reason that there’ll be many others with similar background as you, with the exact same GPA _and_ SAT/ACT scores. You’ll be more of a wildcard to admissions due to lack of information.
If you want to have a chance to get into Harvard/MIT/Stanford/etc you have to take the SAT/ACT. Period.
Because you should make clear if you’re speculating what your saying or know it’s true
This discussion is maddening to me because it assumes that what colleges and universities should optimize for is test performance, full stop.
I've never been a strong test taker, but I gained a lot from my time in college and I've done objectively well in my career. The way colleges, universities, and some of the people commenting here are willing to limit access to education and life-changing opportunity based on one narrow measure is antiquated and narrow-minded.
If I had to give advice to someone who is motivated and interested in higher education but doesn't perform particularly well on standardized tests, it would be to feel no shame in doing whatever it takes to find your way through or around the arbitrary barriers like standardized tests that are going to be put in front of you.
Of course the standardized tests are the best indicator of collegiate performance, and of course you will miss inspired candidates by only looking at test scores. Its a numbers game, and people pay a high price to come to my college, because every student gets a well paying job, because every student has a good grasp of the material and we built a reputation for high standards. If I accept unqualified candidates then my reputation and profit will suffer, because im dumping thousands of subpar kids into the market with papermill degrees.
"Oh no, it was always my dream to go to harvard, its not fair!" Like fuck-all is fair in life.
You probably can get away with it, as long as you don't want to be credentialed and don't want any federal funds.
And that might be the biggest reason to prefer "holistic review" over SAT scores. Originally invented to limit the number of Jews [1], "holistic reviews" is now being used to limit the number of Asians in top universities.
[1] https://www.economist.com/united-states/2018/06/23/a-lawsuit...
See article: https://www.theatlantic.com/education/archive/2013/12/the-fa...
Excerpts from the story: From colleges’ perspective, “holistic” is just shorthand for, we make the decisions we make, and would rather not be asked to spell out each one. It’s a way for schools to discreetly take various sensitive factors—“overrepresented” minorities, or students whose families might donate a gym—into account.
Racist prejudice invades every step of admittance to college, subjective grading, scholarships, hiring, promotion, and firing. Therefore qualifications like education, worke experience, references, and salary history are racist and must be disregard in favor if objective qualifications.
A fair employer would only look at SAT, ACT, and industry certifications that are awarded solely based on passing an exam.
The fact that over the last couple decades employers at large have stopped asking for college GPAs (and by extension SAT scores) on their job applications should instead tell you that all these numbers have very little relevance in the real world.
I got a 1530 by just doing 20 practice tests from the stickied thread on r/SAT and basically learning every possible question type. 1-2 tests a week over the summer and into the early school year. Not a monumental amount of effort and didn't cost me any resources besides time and paper. I think most people just don't know how or what to study.
...oh, wait, we have that admissions system. That's the admissions system of every flagship state school and the US Service Academies. Why does Dartmouth (and before it, MIT's) flip-flop on this matter to 99% of us again?
No one can verify the accuracy of letters of recommendation and teachers routinely break the rules by showing them to students. In some cases I have heard of teachers asking students to write them (!).
> In some cases I have heard of teachers asking students to write them
This is very common in college, most professors don't have the time to write letters of rec for all of the students they do research with etc. They may edit it and sign off after.
Here are just a few insane things.
> Figure 1 shows a linear relationship between SAT scores and cumulative first-year GPA for Dartmouth students. SAT scores are grouped into 16 equal-sized bins, and we plot mean first-year GPA (y-axis) against mean SAT score (x-axis) in the bin.4 Within any smaller range of scores, first-year GPA and the SAT maintain the same linear relationship.
Total nonsense. Figure 1 does not mean that SAT/ACT scores are predictive of academic achievement! Without seeing the variance you cannot make this statement. At all. The fact that there is a linear relationship means nothing statistically.
> SAT by itself explains about 22% of the variation in first-year GPA
So we're explaining less than 1/5th of the variance. This makes SAT scores basically worthless, this account for almost none of the real world performance of people, never mind how redundant they are with GPAs.
The correct title for this would be: SAT and ACT are not highly predictive of academic achievement at Dartmouth.
There's a massive disconnect between the title and the data. The data shows there is likely some statistical effect on performance by SAT. The title would have you think that if you know a student's SAT scores, you would be good at predicting their outcomes at Dartmouth. The data shows this is wrong: if you only used SAT scores, you would almost always be wrong in predicting how well a student would do.
A lot of papers play this game, they imply that statistically significant means meaningful. Nothing could be further from the truth. Something can be statistically significant but inconsequential.
Never mind that they define as success at Dartmouth as your first year GPA.
You are putting words in their mouth from that quote. They didn't use that Figure to claim the test scores are predictive of academic achievement. They specifically say it is evidence of a linear relationship. They argue in the next paragraph that they are and use regression analysis to make their case.
> So we're explaining less than 1/5th of the variance. This makes SAT scores basically worthless, this account for almost none of the real world performance of people, never mind how redundant they are with GPAs.
Compare that to high school GPA, which explains ~9%. The test optional movement implicitly puts more weight on high school grades by putting less weight on test scores.
I'm not saying their analysis is perfect, but your arguments aren't that strong...
At the end of the day, the analysis focuses on a subset of people who are going to Dartmouth. This is a highly filtered subsample of the total population. If you took a random sampling of 18 year olds, gave them the SAT, and then enrolled them at Dartmouth, then I hypothesize you would see a significantly stronger relationship between test scores and college grades than what is observed in their data.
Figure 1 shows nothing. At all. Without knowing if the results are statistically meaningful, showing a linear relationship means nothing. You can draw no conclusions from that figure. The only thing it does is lie with data, to imply to an observer that doesn't understand statistics that this means there is a meaningful relationship between the two variables.
> I'm not saying their analysis is perfect, but your arguments aren't that strong...
They said "SAT and ACT scores are highly predictive of academic performance". This means that they can predict academic performance based on SAT. They absolutely cannot. Their own data shows this beyond any doubt.
I mean... do you have a full double-blind study done when you're sick to determine the right dosage for you, or does your doctor consult an existing dosing study, make sure nothing is substantially different between you and the study group, and then decide upon an appropriate dose. There's a certain point at which over-analyzing makes little sense. The authors here ARE NOT engaged in research. This is NOT peer-reviewed. It's a letter elucidating an opinion by researchers in the field based on their knowledge of previous literature in the field and then a surface-level analysis to make sure that those results ought to apply to Dartmouth as well.
Moreover, you're hyperbolizing yourself. Namely, you criticize the title, which is not the author's title at all. Their letter is titled "Report From Working Group on the Role of Standardized Test Scores in Undergraduate Admissions"
The authors main claim (on the first page after niceties) is that ".Our overall conclusion is that the use of SAT and ACT scores is an essential method by which Admissions can identify applicants who will succeed at Dartmouth." This is not "SAT / ACT scores are highly predictive...". It's an entirely different statement. Let's not confuse HN editorializing with the contents of the letter.
> The title would have you think that if you know a student's SAT scores, you would be good at predicting their outcomes at Dartmouth. The data shows this is wrong: if you only used SAT scores, you would almost always be wrong in predicting how well a student would do.
Except if you read the text, that's not at all what the authors claim. The authors do not claim the title (again... that's HN's title, not the paper's). The authors say:
> In short, consideration of SAT scores allows Admissions to identify applicants who will thrive academically at Dartmouth better than does the use of high school GPA alone.
which is true. Even if you don't think it's a useful predictor, it is a more useful predictor than GPA. This is a reasonable contrast because in the absence of test scores, this would be the only data available.
As for the other critiques. I agree that Figure 1 is not useful and shouldn't have been included. However, the table at the end is pretty straightforwards and provides essentially the same conclusion as above, which is that SAT is better than GPA. Moreover, just because the figure they included is bad does not mean the data they studied was. We'd need to know more, not wholesale reject their conclusion.
The authors then go on to argue that other papers agree with these conclusions (they do), and that alternates to both SAT and GPA (letters of rec, etc) have been shown to exacerbate inequality while having very little power to predict how well a student will do.
> The fact that there is a linear relationship means nothing statistically.
It does... in fact... mean that there is a linear relationship in the observed data with the given levels of significance. I'm not sure what exactly you're going at here. I agree a scatter plot of all the data might be more useful, but honestly, given the plethora of other work on the subject published by so many disparate institutions of higher ed, I really have no reason to believe dartmouth would be much different. So while the included figure is not great, I'm guessing that if we saw all the data the conclusion would probably still be the same.
> Moreover, you're hyperbolizing yourself. Namely, you criticize the title, which is not the author's title at all. Their letter is titled "Report From Working Group on the Role of Standardized Test Scores in Undergraduate Admissions"
The authors clearly say "SAT and ACT scores are highly predictive of academic performance at Dartmouth". That is what they want to show.
And this is totally false. They cannot accurately predict high vs low academic performance based on SAT or ACT scores. Their data clearly shows this is impossible.
> Except if you read the text, that's not at all what the authors claim. The authors do not claim the title (again... that's HN's title, not the paper's). The authors say:
Huh?! That's literally their claim.
They even say in the conclusion "They are significantly predictive of academic success at Dartmouth"
This is just totally wrong. They are a statistically significant but practically insignificant predictor of academic success when you define it in the most absurd and irrelevant way (first year GPA).
> It does... in fact... mean that there is a linear relationship in the observed data with the given levels of significance.
Summarizing data with some line, without any statistics, and saying "look the line slopes up", does not mean there is a linear relationship. At all. You cannot conclude one from the other.
The fact that this isn't obvious is truly frightening.
> They are a statistically significant but practically insignificant predictor of academic success when you define it in the most absurd and irrelevant way (first year GPA).
As I said in my post, the bulk of their argument references other studies at similar institutions
> Summarizing data with some line, without any statistics, and saying "look the line slopes up", does not mean there is a linear relationship. At all. You cannot conclude one from the other.
They run their own linear regression. In no way is the argument "look the lines look linear".
Ironically, I bet most of the US users on this site probably did well on the ACT/SAT too.
I understand that HN has blindspots, but I'm a bit skeptical that HN has a blindspot and a gaggle of Dartmouth professors who were selected to do data analysis have the same one.