The Big Problem with the New SAT | Hacker News Reader

The Big Problem with the New SAT

nytimes.com

32 pointsjack_axel38 comments

There is a fundamental misunderstanding here.

The SAT is not designed to rank students. The SAT is designed to solve a supply/demand problem. There are less spaces available at top universities than there are applicants. The SAT creates a bell curve, allowing schools to go a standard deviation or two out to pick their incoming class.

If the test was an absolute ranking, you'd end up with a large number of people scoring 100%, and the test becomes worthless to colleges as a filter.

The article is making the claim that the test is already pretty worthless to colleges as a filter because the differences that separate, say, a 99.9th-percentile score from a 98th-percentile score (number picked by myself at random, as the article didn't mention any that I saw) don't actually correlate to much of anything of use for selecting who should get into the top schools. Especially if those differences are largely "had more time and money to spend on preparation." It reports that research shows that grades are a better predictor. (Though high schools are also hardly immune to grade inflation pressures, for the same reason that the huge test prep industry has sprung up—parents don't care about merit or fairness, they care about doing whatever they can to get the best-perceived college for their own kids. I find it doubtful that changing the SAT in the way suggested would do anything to address this root issue.)

It also makes an interesting side point about year-to-year results for tracking overall education progress.

On the contrary, "had time and money to spend on preparation" is extremely valuable information to colleges, especially those with ostensibly "need blind" admissions policies.

That is one of the most cynical things I have read in a while. I'm not saying you're wrong, though...

Going through the whole standardized testing regiment last year, I did not notice a need to spend large amounts of time or money on preparation. Although I am sure a few people waste a lot of resources on these exams, many of my peers in the running for top colleges can easily achieve in the top percentiles with little to no effort.

I think the parents contention was that likely in your case and your peers, those resources had already been spent, and that this is a useful signal.

Of course some people do great without studying. I got a perfect score cold, FWIW (~nothing). However, the population of people with great scores also includes a disproportionate number of fairly smart kids whose parents are able to shell out big money for private school and test prep, and those students are a private college's bread and butter. If the schools happen to also pick up some poorer but equally good at test taking students in that population as well, so much the better.

> The article is making the claim that the test is already pretty worthless to colleges as a filter because the differences that separate, say, a 99.9th-percentile score from a 98th-percentile score (number picked by myself at random, as the article didn't mention any that I saw) don't actually correlate to much of anything of use for selecting who should get into the top schools.

Apply multiple filters. The more bell curves you have, the easier this gets. Bonus if those bell curves are not correlated to each other! (I'm not going to comment on how useless this becomes)

Want to be an exclusive college? Use SAT scores to remove 87% of applicants right off the bat. (OK most people below the left half of the curve likely don't bother to apply.)

Now filter out kids who didn't go to enough AP classes. (You've instantly removed children with parents who "didn't care enough" to move to an area with a good AP program! bonus!) Now filter out kids whose parent's didn't have enough discretionary free time to shuttle them to and from multiple extra-curricular activities.

Alright, next up, who can afford to pay for a good essay? Drop anyone who can't.

Congrats, you've just cut down the number of papers you have to read by a lot.

In fairness, once the number of applications become's unmanageable, what else is a college going to do? Spend 1/2 of tuition money on entrance examiners?

"what else is a college going to do?"

Competitive bidding instead of fixed cost tuition. This is what they already select for anyway, this would just be more honest and fair about it.

Nobody seems offended that tuition cost seems to have no relationship with prof/TA salaries, facilities, future income... so removing the charade and opening for bidding seems the most fair.

I suspect the end result is freshie year at MIT will be like $5M to buy a spot, but sophomore year transfer students will probably only have to bid $5K once the less talented flunk out. I pity the instructors trying to teach the wash-out classes. The tests will still exist but for classroom placement, someone belonging in remedial algebra will not be allowed into calculus without passing algebra, geometry, trig, first.

I hate the latest "we only accepted 5% of applicants" measure.

It forces people to freak out an apply to more schools. Which means top schools get more applicants. Which means to accept the same number of people as last year, they have to reject a higher percentage. Which makes people freak out, etc.

When I applied to college, I did 3-4 and thought that was excessive at the time. Now I hear of students applying to 10.

> The article is making the claim that the test is already pretty worthless to colleges as a filter

I'm not convinced this is true. A few years after the writing section was introduced, college admissions were refusing to use it on the grounds that it didn't seem to correlate with any future performance (overall grades, English grades, foreign language grades, graduation rate, etc).

Presumably they wouldn't be saying this if the rest of the test also failed to predict anything.

It's only worthless if the colleges assume that the test indicates a meaningful difference between 99.9 and 98. If they assume that the test is good and not perfect, and use it as a broad filter, then there is no problem.

I am surprised how nobody here seems to have drawn the obvious conclusion here which is these changes (making the test less g loaded) is a massive bias in favour of the rich over the poor. Kids who are naturally smart, but who attend bad schools will struggle with any test that is not heavily g loaded.

Rich/poor is itself heavily g-loaded. What's the net effect here?

It is not that rich/poor is directly g loaded, just that g is highly correlated with income. Quality of schooling is directly related to the income. The more you make the tests reflect past education quality the more you bias it is favour of the rich and dumb against the poor and smart.

> It is not that rich/poor is directly g loaded, just that g is highly correlated with income.

What's the difference? We say an item is g-loaded when it correlates with the latent factor g; this is true whether it's an obscure vocab word on the SAT or whether it's not blowing your income and planning ahead.

I was trying to make point that one of the major reasons income is so g loaded is that education allows the poor-but-smart to rise out of their poverty (and vice versa) via things like the old SAT. Block this and you end up with something like the Middle Ages where the nobility and peasants had the same mean g.

I should say that changes like this are going to have most effect in the middle rather than the extremes. If you are born with a g in the 99.9 percentile band then you will still be likely to escape poverty. It is people born with a g say in the 80th percentile band that will be affected the most by these changes.

This highlights a number of problems with our public education system in general. I think the problems with the SAT highlighted in the article are reacting to these problems, not the other way around as the author seems to suggest.

* There is no standardized high school curriculum across schools, and even when curriculum is standardized, it is taught unevenly. It's hard to have a college entrance exam that covers a significant amount of content as a result, because it puts kids from poor areas at a huge disadvantage in even getting in to college. It's also very hard to identify which students are from good schools and which are from bad ones, so adjusting test scores to compensate isn't a solution.

* Teacher-student ratios in public schools do not lend themselves to the kind of one-on-one work it takes to become a good writer. Unless we're prepared to say "you have a better chance getting in to college because you went to a suburban school district", this can't be a huge part of the test. Writing ability as a high school senior is largely correlated to the amount of attention you got from your English teacher. If you're a good student in a bad school, you probably get very little attention because you're not acting up.

* There's already too much "teach to the test" as it is. Making the SAT yet another test where students have to cram and memorize content is not the solution.

There are plenty of standardized tests that cover content. The problem is that they are far too basic for college entrance purposes. Turning the SAT into a content-focused test does not solve a problem that colleges have; many colleges are not looking for the best students, but for a racially-diverse selection of the most teachable students. What schools should do (and many already do) is require students from lower quality schools to take a semester of remedial classes free of charge to get them up to speed with whatever the base assumption of prior knowledge is at that school.

Writing ability as a high school senior is largely correlated to the amount of attention you got from your English teacher.

There's no way I'd ever buy that. Writing ability is largely correlated to the amount of time the student spends reading... reading novels, instruction manuals, technical books, Web pages, and cereal boxes. Teachers have almost nothing to do with it in my experience, and I think you'll find that any successful author will tell you exactly the same thing.

If you're talking about the ability to write a novel, I agree with you. But you have to learn how to get your thoughts down on paper in a way that is intelligible by other humans, and that's a technical skill that must be learned through repetition and critique. There's a basic technical skill to writing long form content, and many kids don't get it because their teacher doesn't have time to grade 140 papers.

Colleges aren't looking for successful authors. They look for people who communicate well in writing.

I think the top writers didn't need attention from their English teachers, but the fairly good ones probably did.

Actually no for some people its not as a dyslexic I can read very very well but my writing is much worse took me 4 years to get my basic English language O level.

The purpose of the SAT is to be a unbiased barometer; say what you will about it, but everyone takes the same test. Unfortunately, a 4.0 GPA from an obscure high school may tell you very little about a candidate.

I don't know how much admissions officers care about 96th versus 99th percentile, but if someone has an otherwise amazing resume yet ranks in the 40th percentile on their SAT, that's a bit of a red flag.

You can see from class profiles of leading schools that they admit at least 75% of their class from the 99th percentile.

Only reason I got into Stanford is, they make (made?) a point of offering admission to all the 99th percentile each year. I was one of them.

However there're only a few hundred of these nationwide, right? Not enough to fill several leading schools.

3.3 million high school graduates this year, so wouldn't the 99th percentile be the top 33k of them? Seems like that would be more than enough to fill Stanford and the Ivys' classes.

Ah. There were two rankings when I took the tests - math and language. Those ranking 99th in both were offered admission. That was a smaller group.

Interesting. They must have stopped that before 2009 haha.

1983

The problem with the SAT is it mostly judges what you've learned, not how well you can learn. If we're going to have a test for college, it should judge your future potential and ability, not just how well your high school taught you.

The SAT doesn't correlate to grades...

And grades don't correlate to much either.

There's a small correlation between SAT scores and college performance, but that's mainly because the types of questions asked for the SAT are the types of questions that are asked in college exams. SATs have ZERO correlation with success in life though.

"SATs have ZERO correlation with success in life though"

That is wildly untrue for any value of "success" that includes socioeconomic status.

Did somebody get low SATs?

To the extent that SAT scores are used to select which students are admitted to which colleges, you remove any predictive relationship between SAT scores and grades.

I.e. even if SAT scores were perfectly predictive of "college performance" in the abstract, if there were N possible scores on the SAT and exactly N colleges, each of which accepted all students with one specific score, then SAT scores would be completely uncorrelated to college grades. This is an arbitrarily contrived example, but the same principle holds true in the real world.

Grades correlate to either natural ability OR hard work.

More accurately, IMO, grades correlate with:

Current Effort × Current Ability (at the time the grades were earned)

And Current Ability itself is dependent on both natural ability and past effort.

High school grades correlate with college grades. http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/02/...

Reply on news.ycombinator.com