So much for that common, popular notion that standardized tests do not predict anything of value.
So much for that common, popular notion that standardized tests do not predict anything of value.
For GPA, they can hire a private tutor in the subject they're struggling with. There are services out there that will basically write your english essay for you / do your math/science homework.
For college essays, they can hire college counselors to help them draft a compelling essay.
For extra curriculars, they can hire a private coach, etc.
However, there is very limited evidence that SAT coaching actually increases your SAT score. The NACAC did a study on this and they found that average gains of test-prep students is ~30 points (this was when the SAT was 2400 points).
A 30 point increase out of 2400 points is not material to college admissions.
Out of the college essay, gpa, extracurriculars, etc. the SAT is the least influenced by your socioeconomic status. There obviously is an influence, but removing the SAT means more reliance on even more skewed factors.
Here's the study -> https://files.eric.ed.gov/fulltext/ED505529.pdf
You need a uniform grading system, which means a uniform material and a uniform grading process, which is... standardized testing, or at least AP/IB courses.
i still got to go to the college of my choice (fire up chips!) but i have to wonder -- if i was able to boost my gpa using ap/ib courses, would i have received more scholarship opportunities/better offers from other schools?
you could certainly look at past performance of students from that school but that turns into a "legacy system with more steps"...
For one thing, if a school has grade inflation so bad that even an A+ from that school isn't enough to get into Yale - is that a problem?
For another example, if adjustment for grade inflation means Yale will ask for an A+ from Martin Luther King High, Detroit while they'll accept a B from Phillips Academy, Andover - is that a problem?
It's gonna help you when the test writer was playing some gotcha tricks with phrasing or whatever, but if you don't understand the material, even narrowing it down to a 50/50 isn't going to get you a good score, and if you truly don't understand the material you probably won't be able to eliminate half the answers anyway. And they are absolutely aware of the "answer b/c if you don't know" nugget, that's nothing special either.
Also "adaptive difficulty" systems where the system throws harder questions at you after successfully answering the easier ones are basically the "elo rating" of academics. Everyone hates elo but... it slots you into a very statistically accurate ranking. If you score highly on Level 600 questions but you are failing on the Level 700 questions, odds are good you are somewhere between 600 and 700. My understanding is that's what SAT/ACT were moving towards about 10 years ago, that it would be computerized and "everyone's test is personalized" by the elo system probing your exact knowledge level from a bank of questions with difficulty scores dynamically based on how "similar ranked" students performed on that question.
It does miss something: in specific ethnic enclaves SAT coaching is much more effective, perhaps because of a culture of out-of-school schoolwork and teaching beyond SAT prep. Those enclaves aren't particularly wealthy either (if I recall correctly it was a Korean enclave). Even then we're talking 70 points -- not nothing, but also not a radical transformation.
That would depend on the baseline score. For instance, if it was a 1510 baseline and then went up +70, then it would be useful.
If students learn the vocabulary and practice the math to do better on the test, at some point it just becomes the Key and Peele Heist sketch: "That's called a job!"
There's probably another 20 points that can be picked up by learning to read the questions very carefully -- so that you don't race to show how quickly you can spin-up an off-task answer that precisely matches the wrong question. Getting that right also "is not an inherently bad thing."
The verbal section is a bit more of a swamp, and there might be a larger element of gamesmanship there. But for schools like MIT, where math aptitude is the main event, I think keeping a math-focused role for the SAT can help a lot.
It identifies not just the elite-school wizards with lots of AP and math SAT 800s -- but also the teens from humbler public schools that didn't have an AP track, but whose 790s on the (pre-calc focused) math SATs speak to their ability to play at a higher level.
Apropos of analogies, I think the test-takers got rid of those because they can be ridiculously skewed to particular (affluent) cultures. For some people, it's obvious if yacht-to-dingy is akin to symphony-to-quartet. For people who grew up with less money, it is a total WTF moment.
1. Basics: If you don't know standard permutations and combinations then knowing those formulae off the top of your head is nice to have. The language portions in particular, take a lot of preparation for non-native speakers.
2. Speed: Giving a decent number of sample tests helps put you in game mode for the real thing. It also acquaints you to the manner in which questions are phrased and their intended meanings. (big deal for non-native speakers) Lastly, it helps ease anxiety.
3. Gotcha-proofing: Every examination has some familiar gotcha patterns. Some training helps in looking out for them helps.
What I'd do is solve the problem without looking at the answers so they wouldn't bias me. Then look for a match of mine with one of the answers.
As i learned in college, the real "coaching" was rich parents getting rich kids more time on the SAT by getting psychiatric diagnostic classifications that give you more time.
Time has been the real challenge on SATs for most people beyond a certain score threshold, and money can buy time.
The sub-thread and discussion is about wealth bias for exam scores.
The students who are quick and on the competitive math team finish something in 6-7 minutes and some other students are doing correct work but not quite done in 45 minutes. More practice doesn't seem to make them much quicker, either.
And this is in students without a formal diagnosis that allows them to spend extra time.
[There was one time I crashed and burned on a test and ran out of time... where I didn't memorize enough of a big table of identities for a trig test and ended up having to derive everything from scratch]
A typical exam at Caltech would be 4 problems and 2 hours.
I never memorized the trig identities. I simply knew them from using them a lot. And having worked enough algebra/trig problems, you can just see the answer in your head as you read the problem. (This turns out to be a big timesaver at Caltech, where every course was a math course. When you're dealing with calculus, you really need to have moved past struggling with trig.)
At some point in the last 40 years, however, they've slipped my mind.
Sure. But the grandparent's point was: if you're the student taking 3x longer, your parents can buy you a disability diagnosis that gets you extra time.
> I never memorized the trig identities. I simply knew them from using them a lot.
Yah, a reasonable course would make this possible. My analytical trig class was pretty heavy on obscure identities, and the first exams I was like-- no big deal, I know how these are derived, I can figure these out as I need them... For the purpose of that class, nope.
You are assuming the effects of wealth stop at the SAT.
Or the grad students who do the test grading?
BTW, Caltech's testing was done on the honor system. That meant no proctoring, and it was entirely up to the student to adhere to the time limits, and any other instructions on the test.
You didn't need wealth to cheat. Any student could, and with half a brain not get caught.
I recall one physics midterm which 2/3 of the sophomore class failed, including me. I suppose that precludes there being large scale cheating going on.
Not bribe. Hire as tutors with $. This happened pretty regularly at my college (Cornell) where ex-grad TAs were hired as tutors. You could focus on just what you needed to study if you could afford to hire them.
Here in the US we just went thru four years with President Trump (https://en.wikipedia.org/wiki/Donald_Trump) who could not even communicate with clarity and rattled barely coherent ideas. He graduated from Wharton, the most prestigious finance program in the US. He is just one case I thought of. Does everyone really think he was the most qualified candidate to be in the very small inbound class at Wharton? Does everyone thing he was actually qualified to graduate based on merit?
That is indeed where they're moving. They've recently announced that the SAT will be transitioning to a digital, adaptive test in the next 1-2 years. [1]
Notably, the upcoming iteration of the test will only be semi-adaptive, adjusting the version of the second half of the test based on your performance on the first half, rather than adapting to your performance on a question-by-question basis.
I suspect overall this will be an improvement in the accuracy of the results. As it stands, for students with a strong math background, a majority of the math questions on the current test are far too easy and cloud their results on the rest. With the recent removal of the CollegeBoard's math subject tests, high-level math students have very few opportunities to demonstrate their knowledge using a standardized metric.
[1]: https://newsroom.collegeboard.org/digital-sat-brings-student...
AP math and physics exams? Or IB?
True that these are not accessible to everyone though..
If so, that still doesn't imply that rich kids with access to private tutors will necessarily do better on these tests than poor kids - just that anybody with the motivation to read a test-prep book will.
The biggest difference I noticed in how I would take a test versus other people I tried to coach is that I viewed the test as a fun game like a challenging video game level. And those who struggled on the test viewed it as dreadful judgement being rendered on them.
It's like when you can tell someone is extremely self-conscious while dancing: Beyond teaching them any actual dance moves, you have to turn off the part of their brain which is blocking their natural mental resources for problem solving, and that's often the fear they are inadequate to the task, will disappoint their supporters, and that it will hurt their future prospects.
They're doing great at a top 50 university now but it probably was the difference between waitlist and admit at two top 15 schools.
1) The solution to that is to improve security measures. Not to remove the SAT entirely.
2) It is significantly harder to find someone who can score well on the SAT and is willing to take the test for you compared to gaming GPA, extra curriculars, etc.
I've heard of tons of instances of people hiring homework-help services, their dad paying $5k to get their extra-curricular club going, private tennis lessons, etc.
But I've never personally heard of someone paying someone else to take the SAT for them.
I'm sure it occasionally happens, but it's a lot harder to pull off compared to manipulating GPA, extracurriculars, college essay.
Idk about that. I found people to pay me for taking the ACT for them through my alma mater's subreddit. Top schools are full of people who got 35/36, I'm sure there are plenty of other people who scored there and would be willing to take the standardized test for 10k too. In fact, in some ways it's easier to find someone to take the test for you than it is to find someone to boost your extracurriculars because you can structure the payout around the score obtained. I got 10k for a 36, 7k for a 35... no guarantees with tutors and coaches.
Either way, if you cheated your way into MIT, expect to fail out. The hand holding stops there.
I think there probably aren't too many failing out of undergrad simply because they do a great job of filtering in the first place. But I know people who attended that certainly struggled... one who came a C student and still want on to a great medical school and a successful career. There isn't grade inflation.
That depends on your definition of grade inflation. I think most of my undergrad classes at MIT had a median grade somewhere in the B range, maybe B-. Edit: I know some people consider a non-inflated grade curve to be C-centered.
I came to MIT with more than a year's worth of credits from the U of MN, including 6 trimesters of honors level math[0] (multivariable calc, linear algebra, diff. eq.). I had all As, except a B in my Intro to World Politics class. My senior year of HS, I was actually taking a bit over a "full course load" at the U of MN, plus 1/4 time at my HS.
I could sleepwalk through nearly straight A's at a pretty well regarded school's honors program. I was a B/C student at MIT. I like to think that a lot of it was that some "wise" uperclassmen had sat me down my freshman year and explained that once you had a degree from MIT, nobody would ask for your GPA. (The were wrong, BTW. Work for those grades.) I taught myself most of a CS degree while earning a degree in Mechanical Engineering. However, I was also too slow to put my ego in check and admit to myself that I really needed to work hard.
Note that Harvard undergrad has something like an A- average. That’s grade inflation.
That's not incompatible with weaker students being able to pass by with lower grades. Getting bare minimum grades just isn't that difficult.
edit: teammwork also seemed to be strongly encouraged. It wasn't a place that wanted you to fail. It wanted to help you succeed, which is by all means a good thing.
This is pretty common knowledge, in the same way people know that Berkeley and CalTech have tougher classes than Stanford.
It's an industry.
https://www.reuters.com/investigates/special-report/college-...
Not so sure about that. Beyond the fact that that number is an average, the question is from where to where. So many kids get perfect 2400 scores that going from 2370 to 2400 might be the difference of getting eliminated from competitive admission pools altogether. Whereas nobody will care about you going from 1850 to 1880.
P.s. I am dating myself a bit with the 2400 range, which seems to have changed at some point. Transform accordingly :-)
I got a pretty low/average score, but took the test early in junior year so I hadn't taken some of the more advanced math courses yet. I never took it again since I got into everywhere I applied to (didn't apply to ivy league, obviously). Seemed like most other kids I knew did similarly with the smartest kids maybe 150 points higher (2400 time-frame). Nobody I know got a perfect score, or even close to it.
Edit: man, after talking about this I want to see what my score was exactly. No way am I paying $30 for an archived score though. I want to say it was only 1200/1600 (the schools only wanted 2 of the sections). But I'm not sure I trust my memory for something so inconsequential from that long ago.
Extra edit: found my old score report. It's worse than I thought. The writing was 570 (74th percentile) and math was 510 (47th percentile). I'm a lot dumber than I remember.
http://go.collegewise.com/how-many-people-get-a-perfect-sat-...
The mean SAT score is ~1600, so it's a 30 point increase for students scoring in that range.
If you're already capable of getting a 2350+, that means you know everything and it's just down to variance and not making a silly mistake.
A perfect 2400 score is actually really rare. From a stat in 2009, the collegeboard reported that 1 student out of every 5,000 taking the SAT gets a 2400.
The SAT does not operate in the way the LSAT or some other computerized tests work where it keeps giving you harder and harder questions until you start getting them wrong.
I suspect more people could get a 2400 if those who get really close bother retaking it.
"So many" being ~500 out of a population of ~7 million or so
However, I totally disagree that rich people can't game the SAT. I used to be a moderator at /r/SAT and /r/ACT on reddit. All of the questions and answers for all of the exams, including subject tests are known and published online. Both SAT and ACT routinely reuse exams from prior years, and anyone who puts in enough time to study the old exams can do well on the exams. And rich people have the luxury of more time to study, because they don't have to work second jobs or cook for their families or clean the house after school and have more services that can save them time.
Unlike tricks for getting into university like studying after school for classes, hiring private tutors, or taking extracurriculars like lacrosse or rowing, which are great levelers equally accessible to the rich and the poor.
Those rich students, they cheat by studying harder!
Btw, I don't actually think being rich correlates with better academic achievements. It's better to be in the middle, not rich, not poor. To keep motivated.
https://static01.nyt.com/images/2009/08/27/business/economy/...
"This may seem like a counterintuitive claim to some, given the widespread understanding that performance on the SAT/ACT is correlated with socioeconomic status. Research indeed shows some correlation, but unfortunately, research also shows correlations hold for just about every other factor admissions officers can consider, including essays, grades, access to advanced coursework (as well as opportunities to actually take notionally available coursework), and letters of recommendations, among others. Meanwhile, research has shown widespread testing can identify subaltern students who would be missed by these other measures."
n=1
Also n=1.
When I took the SAT it was only 1600 (pre 2400), and SAT prep did in fact help scores significantly.
Back than, the test was designed not for scholastic aptitude (as it's name suggests) but instead to guarantee a standard distribution of scores.
It's been a long time since I cared about the SAT's so I assume once the word got out that the test could be gamed, the people behind it updated it.
> It's far easier for rich students to game GPA, college essays and extracurriculars than it is for them to game the SAT.
Rich students don't need to game anything. They simply get admitted because their family name is on a building.
The mean parental income for Ivy League students is 170k, which is above middle class, but not Bezos-level rich.
What is considered to be rich is a huge spectrum. The difference between 7 figure rich vs. 9 figure rich..is up to a factor of 1000. Those whose parents can donate enough to be commemorated on a building, is an outlier even for the rich. Unless your parents are dynastically rich, being rich is not that much of an advantage for admissions.
Which is precisely what you want in an assessment test. Any assessment test.
You only want a normal distribution if the quality under assessment is normally distributed, but you do want a test where the worst candidate does better than chance, and exactly one candidate gets a perfect score. That's an ideal which is only approximated, but it is the ideal.
At the time, the SAT was purported to provide a score that predict ability to perform academically at higher learning institutions.
Along with other factors such as GPA, and participation in extra curricular activities, a school could reasonably determine how well a student would do.
In practice, the normal distribution for scores correlated with the distribution of college performance. It was a reasonable predictor of success, but it did penalize students from certain backgrounds.
Because the test was devised by psychologists and statisticians, uncovering the pattern to the types of questions and the expected answered allowed test prep people to devise tricks to improve scores beyond the expected deviation.
Sure, a big donation by the student's dad, but that's a known quantity. I took the same SAT you did if it matters.
Which certain backgrounds are you referring to? I'd ask you for the references to show the supposed boost that test prep gives to SAT scores, but then I'd have to find the papers that fail to reproduce it...
Finding data from 1998 online is much harder.
If you can't pay, you take it the one time it's offered for free if your school offers that. Then you get what you get.
We should probably make the SAT nearly free, though, if that price actually keeps people who’d otherwise go to college from taking it more than once.
Q: Could be that being able to "reason calmly under pressure" is something that a future employer might well be interested in?
It might be harder now, I don't know.
Unless you are suggesting adults would buy you beer for you to take the ACT on behalf of a minor they knew. Which seems odd.
This country used to be a lot less uptight.
Not to mention, the drinking age being 18 was for a window of about 14 years in the 70s and 80s. (Unless he's from Louisiana)
I know that one was training to be a dental hygienist, or at least wanted to be training for that. The other few didn't share as much, but they all knew each other so maybe it was the same thing?
I highly doubt I've harmed anyone by enabling their hygienist to get where she was without knowing the formula for the volume of a cone. As far as I'm concerned the gumption necessary to hack your way in is worth just as much as the gumption needed to pass authentically.
Typically they know somebody who knows an adult that will to the transaction with the store and provide the id.
Otherwise, some stores accept good fake ids, or squint to believe that the person buying actually matches the picture on the card
The issue is that the people he claims are getting him alcohol would be in the same situation as him.
I will smile all day thinking about this.
The rich kids in my (public) school were the ones afforded not only tutors, but also just the insight into extra-curriculars. They were the ones who had parents who knew to sign them up for college courses at ~15-16 years old to ensure they could maximize their GPA (5.0 scale, so getting a 5.0 was literally impossible unless you did this). This created a competitive disadvantage for kids who just really didn't realize these resources existed (plus, having parents that would support it + take you to the local college to take additional courses).
Not to mention that, but there were sports our school didn't participate in, but the richer kids had levels of access to that looked great on college applications - rowing and lacrosse specifically come to mind.
So, essentially - rich kids have many easier ways to pad their college applications. It's not that they aren't working hard(er) - they are. But they have easily acccessible opportunities that middle-to-lower-class students (like myself) did not have access to.
--
Coming to the SAT, I'm not surprised by that study. I wasn't afforded the same opportunities as these richer kids, but my SAT score was highly competitive with them. I was ranked 50th in my class with a 4.55 GPA (my 4.0s were gym each year, and I think one or two electives that weren't AP), but my SAT score was a 2300, which was relatively similar to most of the hyper-performant, wealthier kids.
--
This is all super anecdotal. I was definitely upset by all of this at the time - but it didn't affect my life very negatively. I still was able to get into a great school, and have a great career now. But these disadvantages certainly persist against others, and re-adding standardized tests likely will help level the playing field in my mind.
This can be anything. For me, I didn't do sports, or any school extracurriculars. What I did do was run a small business (paper route), used the money to fund my hotrodding efforts, was an Eagle scout (back in the days when that was something), etc.
Basically, you just gotta find something non-trivial to do that demonstrates motivation.
This is worth something.
Uhh, that is definitely not a given
My son is 11 and loves baseball, I've coached a few times as well and it's been a great shared experience. There are definitely kids in Little League / Cal Ripkin who are there because mom & dad said so. But... I've gotten to see my son and a few of his teammates build friendships and mentor relationships with the kids ahead of and behind him that are difficult to do in a school setting.
It's a big deal. When a ten year old stops and is there to help teach an eight year old how to do something, etc those are valuable skills/processes/habits to build. They learn to lose and how to practice.
Part of the "package" a student brings to an application is how they apply those experiences. You can send a laundry list of things, or use your essay/interview to tie it together.
Are you sure about that, especially for an engineering school like Caltech or MIT?
I didn't play lacrosse, football, row, track, baseball, swimming, yachting, nope nope nope.
> Standardized testing was pretty much the only reason I and many other working class folks I know could get into good schools -- I was never going to do a million side activities, and my summers were spent working, not building my academic resume.
– https://news.ycombinator.com/item?id=30833870
> I grew up as a low-income minority in a single-parent household and I ended up getting into good schools pretty much only due to my high test scores, which has been a life changer. Other than test scores, I couldn't afford to do any fancy extra-curriculars.
– https://news.ycombinator.com/item?id=30835611
Of course this is (somewhat) testable. I could ask different collage admission boards to summarize anonymized admission records of students where extra-curricular actives was weighted favorably for admission. And then group these activities based on how accessible they are to various wealth classes before counting them up. I don’t know if that has been done, and I’m not in a position to do it my self... so the best I can do is make a bet.
I had a paper route, which was run as an independent business. I had a territory, I signed up people in it to subscribe, I delivered the papers, I collected the money, I paid for the newspapers the newspaper company dropped off. How much money I made was entirely up to how I operated. If I was sick or out of town, I had to find someone to cover for me.
https://press.princeton.edu/books/hardcover/9780691141602/no...
Review of above book
https://www.mindingthecampus.org/2010/07/12/how_diversity_pu...
My high school had just shy of 4,000 students in 4 grades. My senior year, I took slightly over a "full course load" at the local state university, plus went to high school 1/4 time. Technically, that wasn't supposed to happen, but administrators look the other way for smart kids. I wasn't really competing against others in my grade. People asked if I was smarter than the girl a year ahead of me who went to Harvard. She was my competition. I'm sure something similar happened with a kid a year behind me.
I knew that at MIT, I'd probably just be an average student. However, I really underestimated how hard it is to learn to work hard when you've been able to coast through your first 18 years, despite taking honors courses at the nearby state university, etc. I think the SATs are probably generally pretty good at measuring how quickly students learn, but there's a certain grit it takes to succeed at MIT that the SATs don't cover at all.
Edit: I'm also an Eagle Scout, but I came through after it became significantly easier. It seemed to me that probably at least 10% of the men at MIT were Eagle Scouts. If nothing else, it shows an ability to stick with something for at least a few years, despite it being uncool for most of your peer group.
The shock for entering freshman is very real. I really like the practice of making your first semester Pass/No Record so that there's less pressure to try and get an A, and if you do fail it won't even be on your transcript. Second semester still treats F as No Record as well.
I also got my comeuppance about being "smart".
At the time, being an Eagle wasn't cool anymore, either, and I never talked about it. I was reluctant to even mention it here. Also, these days, it seems that being an Eagle is a project for dad, while the kid is along for the ride. My parents had zero involvement with scouting.
If I were a college admissions officer strapped for time, I'd let the app through on proof of "Eagle Scout" and ignore the other two.
The only easier bet would be seeing the words "I'm Hungarian" on an app for a secret world-saving advanced math project.
Exactly. I went to smallish rural school. We had soccer, basketball, baseball. Youth soccer in my area was a rigged game where only people who were enrolled in the coaches summer camp would make the varsity team. Basketball had 75 kids try out for 12 spots. Baseball is ruthlessly competitive, and while I was a really good little league player, I had no chance against kids who were in multiple travel leagues, etc. I probably played 50 baseball games from age 9-12. The best players played at least 500.
My cousins went to a fancy private school. Everyone played on a varsity team; it was how they did gym. My older cousin did fencing, the middle one did basketball, the younger twins played squash. The squash guys sold themselves as a package and ended up getting a couple of ivy league places fighting for them.
I think we have a similar outlook on this. For me, getting good SAT/ACT scores let me "punch higher", and got me into some really good schools that I would not have be admitted to. Ultimately, I went to a state school, but was able to parlay the competitive nature of things to get a better grant package.
Same thing drove me out of baseball after age 12 or so. I had more than a little natural talent and put in some time drilling with my parents, so I could keep up with the kids doing the traveling leagues and such until (a little before) then. After that it became clear that my parents and I were gonna have to devote hundreds more hours per year (plus not a small amount of money), realistically, for me to keep playing. The gap was just growing way too fast, otherwise. What was left were bad teams/leagues where few players were really trying, so that's no fun, and ones for which I couldn't make the cut. Someone who liked it but just wanted to put in a high-side-of-normal amount of time and effort for a youth sport, had no place.
Living in internet era, I am jealous that nowadays 'poor' students can find so much resources online, most for free, even MIT courses! All you need is an ordinary computer and maybe internet access, which are quite affordable for nearly all families in US.
I will vote up for SAT/ACT and vote down on those EC from a socioeconomic perspective if I have to pick one.
Disagree. I worked for the Princeton Review while in college back in the day. We would outright guarantee 99th percentile for one on one tutoring. If you didn't get 99th percentile the course was refunded or you could take it again. For classroom tutoring we would guarantee some improvement of I believe 200 (out of 1600) with the same refund or take it again option. Candidly, no one would pay for a prep course for a 30 point gain. These course are expensive. Some of them hundreds of dollars per hour.
The cited research is pretty fundamentally flawed.
"Although extensive, the academic research base does have limitations. Most notably, few published studies have been conducted on students taking admission tests since 2000. Only two studies have been published on the effects for ACT scores, and no studies have been published since the 2005 change to the SAT, which added the Writ- ing section among other changes."
This position also doesn't pass the sniff test. GPA is accumulated over four years of study. SAT/ACT is a single test. You can't retroactively improve your GPA by doing better in the past. But you can dramatically improve your SAT/ACT results.
If English is not a student's primary language and fluency improves as they advance academically, up to a plateau with age.
Not sure if this was properly controlled in SAT studies. https://www.ncbi.nlm.nih.gov/books/NBK11007/
Plenty of other domains have study programs with similar guarantees where the program has little to no effect-- it's a simple economic calculation: Set your rates so you make a good profit even if the program has no effect (even if you allow retaking the course, students won't be willing to waste their time and yours forever).
It would be more informative if you knew the actual before/after performance for the program. Elsewhere this has been studied (see links in the thread) and the improvement wasn't that substantial.
> You can't retroactively improve your GPA by doing better in the past.
Indeed, which means that people who's families have been carefully shepherding their education since they were much younger have an _insurmountable_ advantage in the GPA game. For people people who don't come from highly educated families, they only learn about the GPA boosting games as they start thinking about college years to late to take full advantage of them.
They offer a refund or to retake the course. It is indeed a simple economic calculation. The company would quickly go bankrupt if a sizable percentage of students were refunding. The company itself, and the location I worked, had excellent reviews. Even today it has a 4.9 star rating on Google Maps. Quite a high rating for a service that people in this comment section proclaim has no impact.
Intuitively, how could it be correct that people pay thousands of dollars on these courses and receive zero or near to zero benefit from them? If this were true no one would take the courses, the courses would be rated poorly, and the underlying business would fail.
>Indeed, which means that people who's families have been carefully shepherding their education since they were much younger have an _insurmountable_ advantage in the GPA game.
Sure, but the context is "which is easier to game." To game the GPA takes four years of study and prep and you must start doing it at the beginning of grade 9. Tens of thousands of dollars and preparation+work over multiple years. To game the SAT or ACT you must spend a few thousand dollars during grade 11 or grade 12.
"Or check out a SAT book or two for free from the library.", says my partner from an extremely poor family who got into college on the basis of her perfect SAT score and whom never would have qualified to a prestigious school on a GPA basis. (And whom was also admitted to law school on the basis of a nearly perfect LSAT, which she studied for only with free and extremely low cost used materials)
> Intuitively, how could it be correct that people pay thousands of dollars on these courses
People, particularly those to whom a thousand dollars isn't a big deal, spend all kinds of money on speculative and outright ineffective treatments. Including mystical mumbojumbo, quack medical treatments, and products and services which accomplish nothing except contributing to the 'identity' they present to themselves and others. ("I am a parent who cares, look I spent $zillions getting Jr the best opportunities!").
> To game the GPA takes four years of study and prep and you must start doing it at the beginning of grade 9. Tens of thousands of dollars and preparation+work over multiple years. To game the SAT or ACT you must spend a few thousand dollars during grade 11 or grade 12.
I think on this point we're agreeing to a great extent but we're drawing opposite conclusions. I agree SAT improves with focused study, though I believe that improvement is available for free (other than time and knowing you should do it).
You seem to agree with me that it is very expensive to hyper-optimize GPAs, requiring costs and actions extremely early and on a sustained basis.
My conclusion from this is GPA optimization relatively more available to students with more affluent families, because it takes more time, more money, and requires it earlier and more speculatively. -- we don't have an option where you can't improve your performance with the input of time,money, care but we can choose metrics where the available improvement is available to more people.
Because the people paying have a sample size of n=1?
Look, any test will be subject to random measurement noise. If you got a bad night's sleep, got lost on the way to the test, had an argument with your parents or broke up with your boy/girlfriend, you're probably going to do worse. Conversely, if you had a nutritious breakfast or take a version of the test with lots of questions similar to the ones you practiced on you'll do better. Some people get "lucky" (for lack of a better word), while some get "unlucky".
The thing is, unlucky people are far more likely to retake the test and pay for test prep. So people enrolled in you test prep class are NOT a random sample of high school students. The sample is biased to include more unlucky people and fewer lucky people.
People paying for your service don't know whether their child was unlucky or not.
I used the work "lucky" for a reason. It is less likely that those negative factors would affect the student both times they take the test than just one time. So someone who encountered a confluence of unlucky factor for the first test might score higher on the second test simply by having fewer unlucky factors--or even just being more familiar with the test, having taking it previously.
> If this were true no one would take the courses, the courses would be rated poorly, and the underlying business would fail.
Your course doesn't need to actually help anyone. Your customers just need to believe it helped them. How do they know whether you actually helped, or whether the improved test scores are down to luck?
Does your average customer have 20 grandchildren who they can randomly split into test and control groups? Or do they have an imperfect understanding of statistics?
My understanding of the research in this area is that the 30 point gain is if you compare people who take a prep course to people who put in an equivalent amount of time preparing on their own with the widely available (e.g. can check them out of the local public library in many cases) preparation materials.
The 200 point gain is if you compare people who did the prep to people who have not done any prep.
The set of people who can't make themselves put in the time without the structure of a course would benefit by a lot more than 30 points from the course, and would pay for it. And do.
"A survey of NACAC-member colleges unexpectedly revealed that in a substantial minority of cases, colleges report either that they use a cut-off test score in the admission process or that a small increase in test score could have a significant impact on an applicant’s chances of being admitted." (p. 2)
The paper later notes:
"These results indicate that in some cases more than one third of postsecondary institutions agreed that a score increase on the SAT-M of 20 points, or a score increase on the SAT-CR of 10 points, would 'significantly improve student’s likelihood of admission.' This proportion tends to rise as the base level of the SAT score before the 20 or 10 point score improvement rises. This is especially true for the more selective institutions. At lower scores on the SAT scale, a small score increase does the most to improve a student’s chances of admission at less selective institutions; at higher scores, the same increase appears to have an equally large or even larger impact at more selective institutions." (p. 19)
The graphs on pages 18 and 19 give more detail.
The paper also notes that "The College Board gives a specific example of a use that should be avoided: 'Making decisions about otherwise qualified students based only on small differences in test scores'." So it appears that up to a third of the institutions surveyed are not following this guideline. It would be interesting to know who they are.
I agree with your comment about rich kids gaming GPA, essays, and extracurriculars. Daniel Markovits addresses many of these points in "The Meritocracy Trap". Since I don't see how you can prevent gaming GPA, essays, or extracurriculars, given the alternatives, you're probably right that the tests may be better in this respect.
I took the SAT several times. Each time my score went up significantly. My high school ended up creating a new award category for "greatest score increase", or something to that effect. I believe it was ~200 points.
I'd taken a prep course being offered by a local instructor. However, the biggest benefit was from dedicated self-study (Kaplan books, as I recall).
My child did very well on the ACT. All she did was buy a $35 practice book, which almost anyone can do. Literally this was it. Furthermore, she doesn’t know of a single friend or acquaintance that hired a private tutor.
All the nay-sayers belly-ache about this but for most children, even the “rich” ones, this simply isn’t true. It’s most definitely not necessary.
Considering the weight of the SATs/ACTs you would figure a student would at least buy a cheap book and put in two hours per week for a few weeks in practice. In practice, the vast majority of students who do this do quite well. The improvement is dramatic.
Everyone wants to always blame “the system” which yes, has an influence, but nobody wants to put ANY responsibility on the student themselves.
1. They already know the material/are serious and would do fine without tutoring (maybe some small help here and there) 2. They kind of care, but need structure to study. They probably wouldn't study or study effectively without being in a class or having someone for accountability. 3. They don't care and their parents are paying for someone to babysit them to study
Most students were in camps 2 and 3.
It would likely take about 10 contact hours (1h/week, 10 weeks), basically enough coaching so that the result is not artificially low through under-preparation.
Let’s say it costs $100/hr all-in to coach 10 kids. So for $1k you can get 10 applications from motivated, underprivileged kids whose SATs are representative of their ability.
For one $50k annual ride you could run this in 50 low income schools and get 500 underprivileged applicants and then actually admit some of them who might benefit.
Rather than refusing to test, and wasting those resources supporting some kid who obviously won’t hack it.
Your daughter likely has the benefit of strong academic genetics. A child like that who puts in effort (i.e. works through a book) is going to do amazingly well. I'd also argue that families who value academics are also more likely to buy the books to do a couple hours per week in.
She bought only one prep book, for $35, with her own money. I guarantee the very poorest can get that one book, even from the library.
Parents absolutely influence children, otherwise what’s the point of parenting? This is a good thing. I’m just saying the $$ part is way overblown.
The reason people do worse from underprivileged is not only the lack of quality education, but a fundamental difference in the quality of life. Everything from the diet, to available time, and noise pollution, all makes a difference on education level. Our education system relies heavily on outside help, and poorer neighborhoods often don’t have the time and energy to give their kids even the most basic help with their homework.
Well said. Thanks for elaborating on my point.
Parents are the primary educators. When they aren’t educating, or need to educate on too many other things, academics suffer. This is news to nobody. It would be highly suspicious if environment had no influence on educational outcomes.
It probably is the best bang for the buck, if you're already 1-2 standard deviations on general intelligence. Because the test is a game like admissions is a game, so if you're smart enough to see the game, it's easier and most effective to practice with that.
For the "normals", I have no idea if it will work. But we're specifically discussing MIT, who do NOT want normals, I'm not good enough. They want 2-3 standard deviations people.
MIT should be able to weight for socioeconomic and location/environment given the amount of information required for financial aid and "the internet".
If this becomes ubiquitous, I can see suburbanites getting ghetto apartments for the address to game the weights :-)
Look up the crime stats for Honduras and try again.
Also, looking at the very worst situations is not a valid argument. By that rational, what about the poor girl who is kept a sex slave in her basement by her father for twenty years?
Do you think sending your African American example to SAT prep will make a difference? Coming from the poorest and sketchiest towns in America? Such a child would likely be better served by starting at a community college. After that, SATs are not accepted.
I'd rather trust studies than your anecdote. Just saying.
Two SAT tutors have responded to the thread and supported me.
The greatest service that private tutoring provides is structure, accountability, and guidance. A dedicated student working independently through quality practice materials (many of which are cheap or free) can absolutely attain most if not all of the beneficial outcomes of preparing with a tutor/service.
I think when people look at inequity in college admissions, standardized testing ends up being an easy, tangible target, but not a particularly important one. If you want to look at how wealth impacts standardized test scores, focusing on paid preparation programs is missing the larger picture.
Wealthy students have had literate parents who can afford books and have the time to read to them when they're toddlers. They've gone to safe schools where the teachers can focus on teaching rather than making sure the students are well fed. They are surrounded by adults who have gone to college and can serve as role models for positive academic behavior. They have friends who are all taking the same tests and applying to the same schools to provide emotional and thoughtful support.
The collegeboard markets the SAT as measuring preparedness for secondary education. These students have been preparing for college their entire lives; is it any wonder that they score higher?
I went to college. My parents did, as did my extended family on one side. I saw both sides and chose this one.
I picked a town where my children would be surrounded by an environment where cool was defined as “good at school”. I didn’t want external influences contradicting my influence.
My child is dedicated, but I taught that dedication.
A parent doesn’t have to be rich to understand the importance of a good education. My mother came from a dirt poor background.
People never say it’s an advantage to have parents that care about education. They always put the blame somewhere else. The parents worked too hard, whatever. For different traits, some parents are just better than others. Academics is just one trait. Is the most important? No, but it’s one that’s easy to measure.
>My child did very well on the ACT. All she did was buy a $35 practice book, which almost anyone can do.
I don't intend to belittle anyone's accomplishments, but there's a lot more to high ACT scores than a $35 book, no?
In college most of my other "poor" friends that made it either also had a space like that or were taught by their parents to study in public libraries or other spaces designed for concentration.
You basically have someone who's on the board of an elite university coaching kids on their essay, clueing them into extra-curriculars, etc. They get paid pretty well.
I also had a dinner conversation from a lady with two children in Ivy league universities who said she emphasized with the parents who went to jail for bribing schools to get their childrens admitted and would do it herself if her children couldn't get in the school. She also personally knows one of the people doing jail time for bribery.
I had basically no adult academic/university guidance growing up. I just liked reading books in the library and studying things I liked. I was able to receive a scholarship to my university through my SAT scores. I'm not sure how I would square up in the current academic environment when I see the sheer amount of parent involvement in the application process. I also went to a smaller university where the level of tactics and skullduggery is limited.
I'm not 100% sure that the tests can't be coached, but certainly not like the "leadership", etc.
And even if they can't be raised by coaching, the scores can certainly be lowered by poor education and a chaotic living situation.
EDIT: Most people who can pay for coaching are already sending their kids to the kind of high schools that serve to get them ready, so they are close to their peak already.
Even things like summer public service, there are consultants who can tell you, based on your target school, the best one for that school, like is it better to work on a clinic project in Honduras, or teach basic literacy in Burkina Faso.
Never mind that the plane fare to get your youth group to Burkina Faso would pay the school fees for an entire village, with enough left over to pay 1/2 the teacher's salary for a year.
You needed to know stuff as the ante, but knowing the magic bullshit that would give you a good essay score was the key to get the top score. I increased my score ~120 points from taking the PSAT blind in 10th grade to the SAT because I understood at that point that strategy was key and found out about it.
All of this stuff is a red herring though. The nut of the controversy is that standardized tests correlate to IQ. IQ, rightly or wrongly, is perceived to be culturally biased.
IIRC SAT's equivalent of Matura in Poland, so I'll be talking about my case
I've been taking advanced math exam and I had some time + some money (like 10% of minimal wage) during winter break and I decided to buy 3 lessons on analytical geometry cuz I've been terrible at geometry, but since that was analytical, then I've seen a chance to get into that
I've attended those 3 lessons, did some exercises and guess what
on official exam there actually was a task from analytical geometry and I managed to do it and receive full points, which basically increased my score by 10 percentage points (that's a lot, I'd say)
Saying that 10% of minimal wage spent was equal to 10 perc. points is naive, but you get the point
What if I were attending those for whole year? 2? 3? hard to say.
Does anyone know if they have better checks for this now?
> A 30 point increase out of 2400 points is not material to college admissions.
There must be some subset of students who gain much more from "test prep" than others? Even if its benefits for the average student are marginal, maybe there is a certain type of student for whom it is much more beneficial?
Not American so never did the SAT, but I honestly think I would have done much better in high school if I had one-on-one private tutoring. I struggled with focus and one-on-one attention helps keep me focused. Our son is similar – he's gifted and demonstrates his giftedness when the teacher focuses on him one-on-one, but then the teacher has to go spend time on the rest of the class, and as soon as that happens he stops doing any work.
To be fair, I don't think the debate was ever about the quality or predictive value of the tests. There is a small, but well-organized and vocal subset of the population that hates the idea of excellence and differentiation. They want, and have been quite successful in, the replacement of standards of excellence with vaguely defined (defined by them, of course) buzzwords like "equity" and "diversity".
This filters out most people.
I know when we wanted to do a coding test they told use we need to spend 6 months of giving everyone a coding test, have it independently graded by someone not involved in the hiring process. Then after people have worked here for 6 months we examine our actual results from those we hired and see if the tests at all predicted something useful. (or something like that - there is room in the scientific process for some variation)
Google once did a retrospective study and found that interview scores for people we ended up hiring were not correlated at all with people's on-the-job performance. I'm pretty sure nothing really changed as a result of this. I think it's a combination of the industry, especially FAANG, being kind of "stuck" on these kinds of interviews, and a lack of clearly better alternatives (I think there are better alternatives but it's not like I can point to studies backing me up).
> I know when we wanted to do a coding test they told use we need to spend 6 months of giving everyone a coding test, have it independently graded by someone not involved in the hiring process. Then after people have worked here for 6 months we examine our actual results from those we hired and see if the tests at all predicted something useful.
This is interesting but also way heavier weight than anything I've ever heard of. OOC where do you work? (Like vague description of kind of company, if you're not comfortable sharing the specific name).
Big tractor.
This sounds like an unsound result. If you select based on a criteria the correlation with the criteria is usually diminished and sometimes even reversed in the selected sub-population.
Like if you select only very strong people to move furniture then measure their performance. Because they're all strong, you won't observe that weak people are bad at it-- plus you'll still have some people who were otherwise inferior candidates who were only selected because they were very strong, resulting in a reverse result. But if you dropped the strength test you'd get many unsuitable hires (and suddenly find strength was strongly correlated to performance in the people you hired).
Yeah that's very true and I think was part of why they maybe didn't react to it too much. What you really want is to find the people you rejected and see how well they're doing, but we don't have that data.
Still though, naively I think I would have thought that someone who gets great marks across the board should be able to be more successful at Google than someone who barely squeezes by, and I do think it's kinda telling that that's not the case. But I'm maybe just injecting my own biases around the interview process.
edit: This reminds me a lot of this informal study that found that verbal and math scores on SATs were inversely correlated, which seemed surprising, until people realized they were only ever looking at samples all from a single school. Since people at any given school generally probably had ~similar SAT scores (if they were lower they wouldn't have gotten in, if they were higher they would have gone to a more selective school), the variation you see within a given school will be inverse (the higher you do on math, the lower you must have had to do on verbal to have gotten the "target" score for that school).
But as you note, the lack of obvious good alternatives is an issue... and we can't pretend that there isn't an enormous difference among candidates. If we though that unfiltered candidates were broadly similar then "hire at random, dismiss after N months based on performance" would be a great criteria, but I don't think anyone who has done much interviewing thinks that would be remotely viable.
(Though perhaps the differences between candidates are less than we might assume based on interviewing since interviewees should be worse than employment pool in general, since bad candidates interview more due to leaving jobs more often and taking longer to get hired)
I know a fair number of companies that do essentially that. They hire contractors for 6 months, at the end of 6 months the good ones are offered a full time position. The contractor company probably does some form of interview, but they are more interested in their 6 months of overhead from the contractor than quality candidates.
> since bad candidates interview more due to leaving jobs more often and taking longer to get hired
But there are also great people who interview badly.
For Offensive Linemen in the NFL, there is no correlation between weight (which range from 300-360 pounds) and overall performance. A "heavy" 350 pound player is not more likely to do better than a "light" 310 player. But nobody who weighs a mere 250 pounds could realistically make the cut or perform well at the highest level.
For basketball players there is no correlation between height and performance, and there are several standouts examples of players below six feet so there's no cutoff. But if you compare the distribution of the subpopulation versus the general population, you'll see an extremely strong height bias.
Is there such a pushback? As in, is the percentage of the workforce refusing to take such tests increasing?
If anything, I'd say that the proportion of the workforce willing to submit themselves to these tests is increasing, but that's just a guess.
I have terminated interviews before they even got started because of poor interview loop design from employers.
I'm definitely not the person you describe, but the idea of standardised testing being equivalent across all factors just strikes me as being fundamentally untrue.
Personally I am very lucky to test well; and I definitely buy the notion that people who test well in SATs may go on to do better in University, but the reasons are probably the same: freedom from worry about financial circumstances will affect grades. 10 times in every 10.
What's your take on MIT's stance?
our ability to accurately predict student academic success at MIT 02 Our research shows this predictive validity holds even when you control for socioeconomic factors that correlate with testing.
The same factors that lead to success for SATs can lead to further academic success.
I believe that MIT is probably right, in fact, I'm quite certain of it. Many people will drop out of university or perform poorly than their peers for socio-economic reasons, the person working while studying will probably do worse than the person who just studies.
MIT wants the most graduates and especially the most successful graduates, so the institution is right to do this, but I do still think it's more inhumane than I'm personally comfortable with -- but this is part of why I live in Europe where university students in general are seen as an investment by the state and not so much a business to be optimised.
They sure do. So does India. In fact, a lot of other countries rely on testing a whole lot more than the U.S. which has interviews, essays, sports, teacher recommendations, etc.
In this specific case, though, I don't think these two things are in conflict at all. By selecting the best candidates on the basis of merit, MIT is doing what's best for both MIT as well as the broader society.
We all benefit from living in a country that produces top-tier scientists and engineers, and MIT benefits from being a place that is known for producing top-tier scientists and engineers.
MIT's conclusions should cause you to adjust your priors by a certain amount, but they should not cause you to completely flip by themselves -- particularly if you're not in the camp that thinks literally every decision MIT makes is correct by virtue of it being MIT.
If you wouldn't have looked at MIT's original plan of abandoning SAT scores as proof that they didn't matter, you probably also shouldn't look at them picking up SAT scores again as proof that they do matter. MIT's conclusions should lead you to update your priors by some amount dependent on how much you trust you currently have in the accuracy of college admissions processes when they assess student qualifications and outcomes.
----
My personal take on this is that I do absolutely buy that SAT scores could be a leveling factor between kids from different socioeconomic backgrounds and that they could be a better metric than GPA for determining admission. But of course, that's a pretty low barrier of entry to clear, GPA scores are probably close to meaningless when compared across schools. It seems to me that there's a lot of room here for SAT scores to be simultaneously mostly meaningless and at the same time also a reliably better predictor of school success than GPAs.
It's also important to ask what exactly MIT is measuring -- what does it mean by academic success and how much does that definition overlap with "fits in when placed in an environment optimized for people who are good at standardized testing?" And again, even if they are kind of circular or if they're measuring the wrong things, it's still plausible that they're more reliable than GPAs; it's a low bar to clear.
The point in question is whether the students in a lower socioeconomic situation even has a chance to get into MIT.
I was hoping someone had more insight into process or metrics on it.
Your own post has an implicit accusation that lower socioeconomic situations preclude high testing, which has all kinds of implications about the quality of education and living standards as a prior for acumen. I suppose my own bias shows in drawing that conclusion though. As Henry said though,
"If I had seven peasants, I could make seven lords. But if I had seven lords, I could not make ONE Holbein."
If you wish to make a political correct stance, I wouldn't go the money route. I'd say that these kids are victims of intergenerational poverty cycles.
The arguments against these tests are, of course, awful. Objective tests are the best way we know of to remove human bias. Aptitude tests (basically IQ tests) are the best way we know of to measure someone's natural ability (determined in early childhood) with little influence from their experience. Since their arguments make so little sense, it is reasonable to wonder about the psychology of opponents of standardized testing. But their arguments are, at least on the surface, about predictive value.
It is, at its core, a fear that testing largely reproduces the status quo. If one accepts the idea that there is an intellectual elite who constitute the highest strata of society, and that their gifts are innate and heritable rather than trained, it follows that social mobility is pretty much dead. It is a bleak vision.
Personally I think there are different problems that are much bigger and woollier which keep people from non-elite backgrounds down, regardless of test outcomes. The structure of the education sector and employment more widely. Expectations about life and the distribution of rewards etc. We rarely have good quality, nonpartisan discussions about these things which I think pushes people to take views which are instrumental rather than informed.
I have always found the idea of social mobility depressing. It assumes that we will always have a hierarchy, with some people who are powerful and prestigious and others who are poor and always feel inadequate. It assumes that we will always have an underclass but at least people can leave it.
It is. The common argument is that GPAs are as predictive as SATs. MIT says it is not. I think the problem is you only need average ability to a good GPA, but a top 1-5% SAT score confers a higher ceiling of ability. MIT wants to admit exceptional students, not just average or above average ones.
If anything, I would say that my views are the opposite -- homogenization creates a lack of differentiation around skill and aptitude based on questionable science (and sometimes outright pseudoscience) and often leads to an oversimplification of human intelligence in general. It always feels very strange to me that people trying to compress aptitude into a single number say that they're defending differentiation or diversity of talent.
MIT's findings here don't really change my view of the value of SATs, although the findings are interesting and I think they're worth looking into further. I'm not sure "they're more predictive than GPAs" is the glowing recommendation that SAT proponents think it is. You can agree or disagree with me on that point, I'm not here to debate the entire idea of testing or IQ or whatever -- I just want to point out the above comment is a pretty big oversimplification and (in my mind) a borderline complete misrepresentation (I assume unintentionally) of what people like me believe. I can only speak for myself though, maybe there are people out there who do hate the idea of excellence.
One conclusion that MIT hints at (although it doesn't say it outright) is that SATs might be a better indicator of success across economic levels in part because it's harder to buy a better SAT score with money. Looking at things like extracurricular activity runs into many of the same problems as looking at Github repos during hiring processes -- a lot of people don't have time to do a bunch of extracurricular activities, and access to those extracurricular activities is likely highly correlated with socioeconomic status. It might be difficult to move in that direction when access to school resources varies so much between areas.
I do think the SAT could be improved -- I think one really easy way would be to change how it's administered so that it optimizes less for formal test-taking skill. The really good thing about the SAT is that it's a less school-specific measure than GPA. So a better alternative might be a version of the SAT that kept a standardized metric but that either widened its scope significantly or was administered differently.
I also want to put forward the idea that admissions might just be really hard, period, and there might not be an easy way to assess potential, and trying to figure out the easiest way to do it might be like asking, "what's the best way to teach a child to play an instrument in a single day?"
----
One really important point that I want to get across: there is a difference between a measure being good and a measure being "the least terrible option we have at the moment" -- and confusing the two can cause real harm.
At the top of this thread I see the quote, "so much for that common, popular notion that standardized tests do not predict anything of value." And if that's somebody's attitude, then they're never going to find a better option because the whole thing is being approached through the lens of "see, we were right, this is a good metric."
I think a lot of criticism of standardized testing, IQ, coding tests for hiring, etc... is not necessarily trying to destroy everything, it's just trying to point out that many of these measures are really bad and they shouldn't be treated with the respect they're often given. I think that someone can very easily both have the position, "yeah, MIT probably should use SAT scores alongside GPAs" and the position, "people place way too much confidence in these things as an indicator of success."
If that were true, you'd expect countries like South Korea, Japan, and German to perform poorly in science and engineering, among other things.
Diversity may be a worthy goal for societal reasons, but it certainly is not a perquisite for excellence, seeing as there are many highly successful countries that are very homogeneous.
It's wild to me that someone can have the view that the existence of other countries settles the debate over whether or not our school systems encourage well-rounded/successful students given that comparisons to more homogenized schooling environments like China is still one of the more contentious high-level debates about educational quality we have today. Again, I'm not here to convince you one way or another, but that is not a debate that I think most of society considers settled.
> Diversity may be a worthy goal for societal reasons, but it certainly is not a perquisite for excellence
If that's the argument you want to make, then fine, go for it. But then don't say that you're opposing a group that "hates the idea of excellence and differentiation." You are arguing for removing differentiation between different kinds of intelligence and skillsets and compressing that spectrum into an objectively less descriptive metric.
Make up your mind whether I'm arguing for more diversity and more differentiation between people or for less of it.
Colleges are not trying to compress aptitude into a single number. It’s even worse. They are trying to compress aptitude into a single Boolean variable, you are either admitted or not. That’s it. And it seems that subject tests and general aptitude tests are very good indicators of college fit. I don’t know what system you envision, but alternatives I have seen always seem far worse.
> There is a small, but well-organized and vocal subset of the population that hates the idea of excellence and differentiation.
I don't see how that applies to my comment above, and I don't see how saying:
> They are trying to compress aptitude into a single Boolean variable, you are either admitted or not. That’s it.
is doing anything other than backing up what I said. At the point where you are dividing a subset of the population into binary "in or out" groups, you are in fact advocating for homogenization, for less differentiation between students, and for fewer levels/categories of excellence or exceptionalism.
I'm not here to tell you that's wrong, you do whatever you want. MIT is trying to decide who gets into their specific college, fine. But if you're arguing that the point of SATs is to make a binary determination about students, then it's just strictly inaccurate to say that it's the SAT critics who are all trying to cut down tall poppies.
MIT like many American universities does only general admission and that’s indeed would be considered weird in other countries, but it seems like a whole nother issue.
A binary admissions model reduces both. That's not to say a binary admissions model is wrong, but it does reduce vertical differentiation. Of course compressing an integer value into a binary result reduces differentiation, a boolean represents fewer states than a number.
To go a step further, even if that wasn't the case, vertical and horizontal differentiation still can't ever be completely decoupled from each other. Horizontal differentiation allows for greater vertical differentiation by allowing people to vertically differentiate based on their strengths rather than on a questionably representative average of all of their qualities. And I don't think that's a solely Progressive or Left-wing idea, it's a big part of the reasoning behind why economic specialization leads to more advanced societies.
I don't have a single-comment answer to replacing the entire SAT and reworking the entire college admissions process, and it's feasible that the SAT might still be preferable to pure GPAs in the meantime. But I don't think saying that requires us to pretend that compressing skillsets into an objectively less granular/descriptive metric is a good thing or that it's somehow increasing our understanding of student skillsets. Saying that MIT might be right to accept SAT scores doesn't mean we need to pretend that the SAT doesn't have very serious flaws. Certainly it doesn't require me to pretend that every argument against SATs are arguments against meritocracy, I think that's just objectively wrong.
Ideally we would have standardized metrics that were more granular, and ideally we would at least have an SAT that was administered differently and more regularly so that they were optimized less for formal test taking skills. But there are a lot of barriers in front of that.
----
I also don't have a single-comment answer for what to replace Github repos with during hiring interviews, or how to make whiteboard coding tests more accurate, and I have criticisms about them too. The answer might be that there isn't an easy single number that represents meritocracy, and we might be fooling ourselves pretending that there is, and it might just be wishful thinking in the first place to pretend that there is a version of admissions processes for colleges that isn't fiendishly difficult and complicated and multifaceted.
When people criticize whiteboard interviews on here, it's reasonable to ask if there's a better system, but I rarely see people saying, "you're only criticizing whiteboard interviews because you hate meritocratic job placements." No, I have criticisms of these systems because they're not good representations of talent.
Except no one goes to MIT because it's "diverse", whatever that even means anymore. They go there because it is one of the best schools in the world.
1: https://www.collegefactual.com/colleges/massachusetts-instit...
2: https://www.collegefactual.com/colleges/california-institute...
To people with a particular agenda -- that society will be improved if equal outcomes are mandated -- I suppose standardized tests aren't "valuable" to them.
I am very grateful for the SAT. I wasn't a good student. I was unable to do any work outside of school because of a bad home (my brother did hours long "hand-clapping/stimming" and chanting rituals, my mother drank, and my father was violent). But I did well on the SAT -- enough to get a national merit scholarship and a scholarship from Hofstra University where I got my BA in Math in the early 80s.
If it wasn't for the SAT I don't know what I would have done. Standardized tests are the only answer, and I was very upset when I read of schools getting rid of using them for admission.
SATs are also a great predictor of a person's ability to complete college. That was one of their original uses. So without SATs, you'll get more people doing poorly, and more people with no degrees and a lot of debt. And if these people are members of what's considered an "under-represented minority" then there will be even more remedial action required elsewhere to fix the problem of the high failure rate (like giving them degrees anyway, etc).
What would be a more interesting measure of real value is to study for academic innovation and invention (valuable to society), as well as future success on the job market or at new business ventures (valuable to the student and economy).
I'm making the assumption here that since we have limited educational resources, we'd want to provide the best education to those most likely to advance an academic field through new discovery or insight or invention, as well as those who'd best innovate or provide for existing business and services to society.
And I'm curious if we've ever had any study looking into that? Or if this one did?
When I took the ACT in 2006, I scored around a 24 or 26 for composite score. Not very good. I didn't prep nor study for it because I was pretty apathetic about school and did my best to coast on whatever natural talent I could muster. Since my score wasn't very good, I retook it several weeks later. About halfway through I realized I was given the exact same question set as the first time I took it. I had of course not studied nor prepped for the second time, being the apathetic teenager I was. However this time I score around 32 or 34 or so. I don't remember exactly.
What was different? Why did my score go up about 8 points? Better mood that day? Which score was the "real" one that best represented my abilities?
Did I eat a better breakfast beforehand? I recognized some of the questions but of course I never expected to see the same questions a second time so I didn't prep for that. I didn't prep for anything about it.
I wonder how many kid's college admissions results are ultimately because of one bad or good day? I suppose the other lesson is if you get a bad score, try again. You only need one good score.
It is nice to see MIT do justice to those priors.
That being said, they seem to have backed up their numbers, and MIT knows how to count, so they must be right! I just always hoped SAT/ACTs weren't that conclusive so that we didn't have to go through them anymore and could focus on the funner AP/A Level stuff :)
That said, outside of admissions, I don't think I got academic value out of them. They were hard for the sake of being hard. I'd rather have taken the SAT or ACT any day.
(also apropos of nothing but I don't think much of the writing section on the SAT either, which was a hot topic 15 or so years ago... a huge amount is dependent on the graders, and it's fundamentally a "blackboard programming" type scenario where the student is separated from basic resources like word processing and graded on the resulting product... that's not how you would actually work in an academic setting.)
I went to a barely-known state university and was very surprised when some of my intro-level gen ed requirement classes mostly covered material I'd already seen in, and with a similar level of rigor to, junior high school. And my junior high and high schools were nothing special at all—at the higher end of performance in the state (so far as those measures are helpful, anyway) but just regular public schools in a state with overall mediocre-bordering-on-poor schools.
If I'd known that the first couple years of college weren't going to be harder than high school, and would have a lower total time commitment, hell, I'd have probably tried to go the drop out -> GED -> start college at 16 or 17 route. I wasn't gonna get into top-tier universities, anyway.
That isn't what they said. They said that access to those tests is not universal. Students from high schools that don't offer AP classes would have a hard time taking AP exams. This would exclude people from rural or impoverished areas.
This is why the SAT and ACT are useful: they are meant to be aptitude tests. They are IQ tests in disguise. If properly designed, they will measure intelligence with minimal influence from education or cultural background. Theoretically something like these tests could be administered to elementary school students and still be useful for predicting success in college a decade later.
Yeah, I wish they'd just flat out told me "we expect AP courses" before I applied for MIT back in the day. Would have saved me a lot of hassle that just resulted in "sorry, we wanted AP credits" in the end.
I aced the SAT and ACT but had a decent number of mediocre AP scores because I was seeing the material for the first time when I opened the test. Got to college and after a single 45 minute lecture covering the gap material, I'm pretty sure I could have scored a 5. Ended up making for several easy A's freshman year.
> To be clear, performance on standardized tests is not the central focus of our holistic admissions process. We do not prefer people with perfect scores; indeed, despite what some people infer from our statistics, we do not consider an applicant’s scores at all beyond the point where preparedness has been established as part of a multifactor analysis
It was always an example of people refusing to believe something because it would be nicer if it weren't true.
If there was a test that could accurately predict someone's chances of "success", be it SAT or IQ test, it would be used by everyone for everything. Billion dollar companies would be giving CEOs a version of that test before hiring them. A near perfect SAT score is noteworthy personal trivia, but other than that it loses all meaning as soon as someone steps on campus.
Schools with strong brands use tests because they have way more applicants than they can properly review. Standardized testing is made necessary by scale, not predictive accuracy. I'm sure that anyone who has worked in college admissions for years has a very clear picture of what a successful student looks like. But there doesn't seem to be a way to quantify and codify that knowledge. And there likely wouldn't be time to apply it to tens of thousands of people in a few months anyway. So, standardized testing is what we have until someone comes up with something better.
You really should look into the g-factor research. This isn't about knowledge, but rather performance. It turns out that your performance on geometry and vocabulary tests is highly correlated with your performance on tests in such disparate fields as Classics and Music:
https://en.wikipedia.org/wiki/G_factor_(psychometrics)#Cogni...
The crazy result in general intelligence is that your performance in all these areas is highly correlated, and incredibly correlated with career success:
> "Research indicates that tests of g are the best single predictors of job performance, with an average validity coefficient of .55 across several meta-analyses of studies based on supervisor ratings and job samples."
In terms of gaming scores on standardized tests generally, yes people with more money can take prep classes and the like, but there are also test prep books available--presumably even from the library--that probably get you a lot of the way there.
It's backed up by research, for example: https://news.uchicago.edu/story/test-scores-dont-stack-gpas-...
I don't doubt that MIT's study showed otherwise for their needs. But UChicago is also a top-tier school and is not requiring standardized tests, for educational not political reasons.