Comment by rayiner | Hacker News Reader

Comment by rayiner | Hacker News Reader

Parent Full thread

The SAT isn’t really coachable for most people: https://www.wsj.com/articles/SB124278685697537839

> The college counselors' report concludes that, on average, prep courses yield only a modest benefit, "contrary to the claims made by many test-preparation providers." It found that SAT coaching resulted in about 30 points in score improvement on the SAT, out of a possible 1600, and less than one point out of a possible 36 on the ACT, the other main college-entrance exam, says Derek Briggs, chairman of the research and methodology department at the University of Colorado in Boulder and author of the admissions counselors' report.

https://slate.com/technology/2019/04/sat-prep-courses-do-the...

> Once scholars control for all these factors as best they can, they find that coaching has a positive but small effect: Perhaps 10 or 20 points in total on the SAT, mostly on the math section, according to careful work by Derek Briggs of the University of Colorado Boulder and Ben Domingue of Stanford University.

> Some estimates run a little higher: A 2010 study led by Ohio State’s Claudia Buchmann and based on the National Education Longitudinal Study found that the use of books, videos, and computer software offers no boost whatsoever, while a private test prep course adds 30 points to your score on average, and the use of a private tutor adds 37.

> The SAT isn’t really coachable for most people

Based on info from a buddy of mine who runs an SAT prep business, this is a half truth. He says:

- It is tough to get gains from a kid who doesn’t want to be there — that is, the parents are pushing the students to prep when the student doesn’t want to. My friend actually tries to filter out these folks in his sales funnel.

- Motivated kids can get ridiculous amounts of improvement (40-100 per skill), especially if they are starting off in the lower or middle part of the scoring range (people at the top are capped in improvement by max scores).

- I have done some minimal tutoring for a friend’s kid, and his SAT score went up by 110 in math and 80 in verbal from a middling starting point. Anecdote, but still... My “curriculum” was just have him take a practice test every week under test like conditions, review the principles behind the math problems that he missed, review any test strategy issues that pop up, and give him 50 vocab words a week with skills on how to search for how these words are used in a natural setting. With a longer runway, I think he may have been able to double that improvement (the kid was smart enough to get a perfect math score, but didn’t have the confidence to execute in a testing context).

I don't know why you're getting downvoted. You are correct. It's very much possible to improve scores by practice. By simply taking a lot of practice tests over the summer, my daughter was able to substantially improve her ACT score from 30 to 34 - which is in fact what she got in the actual test.

The exam is totally coachable, you just have to do cram school. Those studies look at private tutoring which spends possibly 1/20th of the time kids spend in cram school. Cram school is often conflated with private tutoring but they are completely different things.

Sure but the amount of time you have to spend on cram school starts to make it not worth the points gain. The top schools all reject a bunch of 1600 scorers every year. If all you have going for you are good grades + good SAT you probably aren't going to get into an elite place. Sure it'll help you get into a tier 2 school, but at that point you're usually better off with a public option.

That’s the point of the entire argument. Elite schools don’t go by scores and grades alone.

Parents who sent their kids to cram school, or tutoring or whatever, and kids with perfect scores are mad that schools aren’t basing admissions solely on scores and grades.

What it boils down to is that Asian American kids are outperforming most other groups on the SAT for whatever reason. Elite colleges don’t want student bodies that are 70% Asian American.

Elite colleges also don't want the type of people that will spend 4 hours a day cramming for an uninteresting test.

Yes, but elite public high schools do. Which feed the elite colleges.

Also, middle schoolers don’t really have a choice in the matter. If it were up to them, they’d play Roblox and watch TikToks all day.

No they probably don’t. That’s one side of the argument.

A single second is a very small amount of time, but in a race it can be the difference between winning and losing. When you're in a competition with a thousand other people whose scores are almost the same as yours, 30 points can be huge.

And it should be noted that they changed the tests before these studies too.

> Many of the key studies of coaching effects were based on kids who took the SAT or ACT in the 1990s and 2000s, Briggs told me. The tests have since been changed a fair amount, in an effort to make them more like statewide achievement tests and to minimize their similarity to intelligence tests.

I am an outsider to that whole system, but when the SAT is not coachable, then what is the purpose of schools?

This talks about coaching for SAT specifically, as opposed to general education.

For example, looking at specific question formats; learning the only topics that would be covered by tests and nothing else.

Specifically 1-3 hours a week for a few months isn’t enough to dramatically improve the average score.

That’s all the referenced studies show.

That doesn’t mean that the SAT isn’t “coachable” with much intensive efforts, or that more minimal efforts don’t result in large improvements for some subset of test takers.

Just a quick note than your analysis (here, I don't know about elsewhere) depends on the effectiveness of test prep courses, but those aren't the only advantage well-off student have in the SAT; other advantages, all of which might be more important than coaching, include stuff as simple as "expectations and forcing functions to take the SAT more than one" and as lurid as "phony IEPs to enable the taking of the test over multiple days", which is a thing.

"The SAT isn't gameable" is an extraordinary claim. And "suburban families try hard to game the SAT" is not so much a "claim" as "a fact your own analysis acknowledges". I could be persuaded, but what you've presented here doesn't constitute extraordinary evidence.

(Your other argument in this thread, about the possible Cobra Effect of eliminating the SAT, is much more persuasive).

1. The results you list are just average. There could be a bimodal distribution.

2. What those studies call coaching isn’t remotely comparable to years of 20 hour a week cram school.

3. Of course the SAT is coachable. How do you think you learn the math techniques and vocabulary in the first place?

If it’s not possible to increase your vocabulary, how do people learn new languages?

4. The percentage of students who make a perfect score on a similar test, the ACT, has increased by 5x in the last five years while the average score has only risen slightly.

Clearly a subset of test takers has figured out a way to reliably prep for the test.

Well what they do is move people from the 96th to the 98th percentile without a meaningful increase in skill or attitude. The bigger issue is that schools have their heads up their asses when they insist on only selecting from the 99th percentile, when slumming it in and going down to the 93rd would be more meritorious and not negligible to their graduation rates.

Reply on news.ycombinator.com