SAT scores can have impact beyond college | Hacker News Reader

SAT scores can have impact beyond college

post-trib.com

17 pointstokenadult56 comments

Attention those of you only skimmed the article or haven't read it at all: pay attention to this piece at the end: "Gerald M. Bradshaw of Crown Point consults with students on how to gain admission to selective colleges, universities and law schools." In other words, the author of the article has a tremendous interest in making you think that SAT and other scores are important, and he cites no hard data indicating the growing trend he alleges he exists. Those of you who voted this up probably didn't read PG's essay, "<a href="http://paulgraham.com/submarine.html">The Submarine</a>", which covers just this kind of thing.

Stories like this make me wish I could vote articles down.

Sorry to reply to my own post, but this relevant quote from The Submarine is worth considering:

"A good flatterer doesn't lie, but tells his victim selective truths (what a nice color your eyes are). Good PR firms use the same strategy: they give reporters stories that are true, but whose truth favors their clients." In this case, however, I'm not even convinced we're getting the truth.

You have done something more useful than voting the article down by expressing precisely why you think it is a suspect source. Thanks. I submitted the article figuring I would get some well-informed replies about its accuracy here.

After edit: Note that some other replies are indicating that some current employers ask job candidates for SAT scores. That has never been my experience, which is why I wanted the reality check on the submitted article. In some hiring processes, I could have gained ground against other candidates if SAT scores had come up in the hiring process.

But did you not even notice the bit at the end? Halfway through, I figured this guy was selling services, and went to the bottom to find out what his affiliation was.

I was aware of other workplaces, from other accounts by persons not in the test prep business, where SAT scores are a hiring criterion, so I thought this recent blog post was a good enough source to bring that issue into discussion.

I recently had a phone interview stage with some consultants who asked about my SAT scores (note: I graduated college in 2002).

After following through and discussing other aspects of the job with them, I'm now inclined to give out low SAT scores when asked. It will be a great filter to avoid interviewing at places I would hate to work at.

Agreed. I would actually laugh at someone who asked me for SAT scores. I'd want to see actual research documenting that SAT's are actually a good predictor of job performance in the long run. (Hint: they arent). In the consulting field, I would expect people to do their research before they pick arbitrary metrics of selection. They might as well ask you your favorite university or sports team. That (might) be barely classifiable as a culture fit...

Its really amusing to me that companies use this as a valid benchmark. Especially, as the article states, for SALES of all things.

Configuresoft Inc., a Woodland Park, Colo., systems-management software company, requires combined scores of 1,400 in CR and math for applicants applying for sales positions. A spokesman for the firm said the company's clients routinely demand salespeople with high test scores.

I declare shenanigans. When (in the course of a sales discussion) does the salesperson's high school SAT scores come up? How about never.

If anyone knows these guys or has seen these practices, I'd love to hear the actual basis for this from an HR perspective, since I dont believe the research on the topic puts SAT's anywhere near predictive, especially in sales.

You still need to be smart to be a good salesperson, and what this article is claiming is that you can't really compare grades at one highschool or college to grades at others, but the SAT is one universal test given the same way across the country. Its biased and imperfect, but at least its comparing apples to apples.

That said, I'd also like to see research on SATs as a predictor of job performance, either way. You say they aren't...do you have a link to some research finding that?

Unfortunately, I dont have any easily referenced research in front of me or easily found. But I did take a significant number of HR classes, and I remember those professors railing against industry for horrible, unscientific decision making metrics across the board. Unstructured interviews are by far the worst predictive methods, but SAT and IQ are very poor predictive measurements.

They measure intelligence, sort of. In fact, they don't really even measure intelligence that well, becuase you can heavily study for them. Even setting that aside, intelligence is also a bad metric for job performance in most fields. Motivation, Past Job/Work Experience or Work Sample tests, and Psychological factors are much more predictive.

Anecdotally, I'm sure most people that can think of very intelligent people that are very bad employees. And also some that are very good employees. And plenty that are average employees.

---------

EDIT:

After digging out my books, I'm dead wrong. :) Willing to admit it. I think my memory was confusing job satisfaction with job performance. From one of my textbooks, as a summary:

Mental ability tests, and those been called intelligence, or IQ, tests are the same type of tests. The term mental ability makes explicit that these tests measure various cognitive abilities of the applicant. That is, these are abilities that primarily indicate the individual's level of mental manipulation of words, figures, numbers, symbols, and logical order. Following from this, it is fairly easy to understand the strong relationship between mental ability test scores and academic performance. Formal education primarily stresses cognitive exercises and memorization of facts, and these are the components that make up a large part of most mental ability tests. This however, does not mean that mental ability is useful only for academic selection. Many jobs in organizations demand the use of mental abilities, including managerial, technical, and some clerical jobs to name but a few.....Only one of these devices, work sample tests, has a higher validity than does mental ability tests.... What can we conclude from all these data about the use of cognitive ability tests in selection? Two factors must be taken into account when answering this question. The first is that evidence seems to be overwhelming that these tests are among the most valid of all selection tests for a large number of jobs, some selection specialists would say for all jobs. The second factor is the evidence that these test exhibit adverse impact...among demographic groups. But becuase these tests are also valid, there is a good chance that their use in selection could be defended against claims of discrimination. Therefor, some have thought that cognitive ability tests should be used extensively in selection. ("Human Resource Selection", Gatewood & Field, 2001)

I stand corrected, by myself. :) Unfortunately, the research I'm reading through doesnt specifically address SAT as a method, just 'intelligence tests' in general, and has some industry-specific mental/cognitive ability tests. For those interested, this category includes things like the Wonderlic, the Minnesota Clerical Test, and the General Aptitute Test Battery. I'm still curious about how well this translates into Sales, but that will be a personal research project for later date.

After digging out my books, I'm dead wrong.

Commendable of you to say that here on HN. That's a good example, one I will have to follow whenever occasion demands.

I had thought that there was some supporting research for the idea that cognitive tests are a reasonably well validated job screen, but I didn't have references at hand, so I'm glad you went to the trouble to look it up.

Both IQ and SAT are time constrained and prohibit the use of third party reference materials and other people. That's very different from work environment: you can stay after hours or come in earlier and most importantly you work with other people and with all sorts of reference materials available.

Work Sample tests

I've always liked hiring processes that involve producing a sample of actual work. I usually fare well in such processes when I'm seeking jobs I would enjoy. Alas, some jobs are harder to boil down to a sample than others.

After looking into this for about 15-20 minutes, using basic google scholar and some other academic sites, this appears to be something that is actively being debated in academia -- whether intelligence testing is valid as a predictor or not, and there seems to be competing recent research in both directions.

Unfortunately, without access to the full articles, its pretty hard to gauge anything since abstracts are pretty trivial.

I applaud you for your restatement.

It is very surprising to me that scoring well on tests is in an indication of job performance. Maybe it has less to do with intelligence, and more with the desire for conformity. After all people who do well on standardized tests are often driven in a very conventional path.

I would tend to agree. Unfortunately, the one thing that is somewhat above this discussion is the idea of assessing job performance itself.

The research I'm referencing links hiring metrics to performance reviews. Clearly, there is something to be said that the items measured in performance reviews might be highly suspect as well, and prone to many forms of bias -- so earlier methods that predict these could actually be just self-fulfilling, not actually a good way to find awesome employees in any particular situation. Again, unfortunately, thats a little beyond the scope here, clearly.

In addition, I think that people that did well on standardized tests tend to be for their usage in hiring metrics, and those who didnt but consider themselves successful do not. Bias at work.

As an ex-SAT tutor (horrid job, but while in school it seemed like a good idea...), my take on the matter: high SAT scores don't mean a damn thing, I've seen absolute fools hit high 700s across the board, but low scores can be telling, especially in math (verbal is a lot more variable, from what I've seen, and also easier to game through prep; math is harder to "fake").

Someone going into a technical field should be able to do well on the math section, and I would not want to work (programming, at least) with someone that could only manage a 450. I've never met a person that couldn't get at least five or six hundred points on the math section that I would consider even remotely qualified to go into a science, math, or programming.

Frankly, the best people should (and usually do) do even better, at least on math. The material is just too close to what you've spend years doing in school to use the "bad at tests" excuse - if you have any aptitude for quantitative reasoning at all, you will not do poorly. Most of the smartest people I know (granted, mostly acquaintances from physics, so it's a biased sample) hit 800 with ease.

So overall, I can see some use for the SAT as far as trimming bottom of the barrel candidates. But the difference between a 650 and a 700, or even a 750 and an 800? Meh...that's maybe three or four questions. A person's score varies by that much between different tests, so it's utterly irrelevant as a fine discriminating factor. The difficulty level of the test is _far_ too low anyhow to discern a good candidate from a great one.

Interesting points you bring up and Google effectively came to the same conclusion when they were smaller (low hundreds instead of tens of thousands of employees). Everything at Google is data driven, including the hiring, but things have changed drastically. Back in the day, Larry and Sergey had a forumla that used the SATs and your graduating GPA. The theory was that your SAT was a measure of your potential, and your GPA was the ability to deliver on that potential. As you can imagine that ultimately ended in failure, but kind of an interesting idea none-the-less.

Since then they've wound up taking a much more Googly approach to hiring, by collecting tons of data from their employees. They effectively gave everyone a survey that attempted to figure out what defined the Google culture and success there-in. By hiring people consistent with this data, the theory is that they can better keep the culture and continue to hire successful engineers. Lots of interesting tidbits came from the study, but the most interesting was that a masters degree will (statistically) always outperform a b.s, but a phd is closer to a 50/50 for outperforming a masters (they attribute this to being too specialized. I forget the phd-to-b.s. probability).

All this is from memory and is a couple of years old, so take it as you may.

This is not a new thing that google invented (although, its possible they did it much better). According to some of the things I've read today, one of the benefits of using intelligence tests is that they tend to be valid across industries and positions, which means that unlike many other forms of hiring selection, you do not need to do an internal HR audit for applicability. This makes them convenient for enterprises of all sizes.

However, in the past, and currently for items like psychological / social fit / performance metrics, etc, it was usually necessary for a company to find their good hiring metrics by doing an audit of previous 'successful' vs ' unsuccessful' employees, and compare attributes. The successful employees made up the desirable attributes, as you might expect, and this is what HR looked for. There are many, MANY problems with this way of thinking, but it did work fairly well, I think. This has been done for a long time, and the army started it.

It seems likely the one thing google could apply to the process is its significant algorithmic capabilities.

i don't think it's that ridiculous. if i'm a recruiter for a consulting company (or any type of company), i want to know if you're smart. i can figure it out by talking to you, but that's limited by your ability to bullshit and my ability to figure it out. SAT scores can give them a little more information about you.

only stupid recruiters hire based primarily on SAT scores. It's silly to say that SAT scores should have zero influence. if you were hiring and saw two identical candidates (or more realistically, two candidates that, considering everything else, you thought were equally qualified), would you choose the one with the better SAT score, or flip a coin? Asking for SAT scores only takes 5 seconds.

How do you measure an entry-level candidate salesperson otherwise though? In CS you can ask theory questions and ask them to program something for you. In sales you... ?

I agree it may be a lousy metric, but that doesn't mean it's not the best one they have.

Traditionally in sales you hire a ton of candidates, give them hefty performance goals right out of the gate, and then fire over 80% of them in the first month. Rinse and repeat until you have your organization staffed.

It's a brutal field.

We're adding a little something to this month's sales contest. As you all know, first prize is a Cadillac Eldorado. Anybody want to see second prize?
...

Second prize is a set of steak knives. Third prize is you're fired.

http://www.imdb.com/title/tt0104348/quotes

As another poster mentioned, sales is very performance oriented. Either you make your numbers, or you dont. This sounds like a bad way to hire, but actually it works very well and is fair.

Also, per your own example, they ask you about theory questions. This is, in some sense, a job-sample test. Many developers are asked to write code during an interview. That is a job-sample. Salespeople are often asked to sell something arbitrary. Job-samples are VERY valid predictors.

Hypothetically, just interviewing the candidate at all should be enough, since a job interview is just an opportunity for the candidate to sell themselves. If they're good at sales, they'll sell themselves and make you want to hire them. Interviewing CS folks is hard in comparison because selling yourself in an interview is a talent that has almost nothing to do with making software.

I am slightly interested in finance, mainly quantitative trading. While looking into the field, I found that the more selective banks and hedge funds asked for SAT scores.

I got the impression that the only companies that really care about SAT scores are the more prestigious ones, and those generally used it as a filter. If HR is interviewing top students from the ivies or MIT, I doubt a score at or above the 95% would be extremely impressive. I'd be surprised if most companies cared.

I did find one article talking about hiring the link between SAT scores and a fund manager's annual returns.http://www.quantnet.org/forum/showthread.php?t=1891

I have put test scores on my resume. I did decently well, so so my scores show some indication of intelligence. I don't really think most companies are very impressed by high SAT scores though. But I would doubt people would see a decent SAT score on a resume as a negative, and some companies care about SAT scores.

Anecdotal, but recently one of my old college roommates who I remember had a score of somewhere in the 1400-1500 range tried his hand at a sales job. I think he lasted 3 days before getting fired. Honestly, thinking back through my college best friends (the only group of friends whose SAT scores I ever knew) I think our sales ability would be in inverse order of SAT scores. The one that had the lowest scores currently does non-profit fundraising (meaning she regularly pulls in donors signing tens to hundreds of thousand dollar checks).

Doesn't IQ correlate well with job performance? Don't you think the SAT predicts IQ fairly well?

If this is actually true, it is yet another consequence of the infamous court decision banning IQ tests in the hiring process:

http://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Company

Want to know why you had to go up to your ears in tuition debt? That's why. College has become an expensive IQ test. And now, even college is in the process of being watered down into meaninglessness as a qualification.

There is much truth in this reply. I don't know that I buy into the assertion of the late Richard Herrnstein that all employers of ALL kinds of workers would do well to screen job candidates by IQ tests, but the law now in the United States is that they can't. Absent a specific validation study for a specific job, employers have to take care about having hiring requirements with "disparate impact" by ethnicity. Requiring postsecondary education that is not strictly necessary for the job gets a free pass, legally, but requiring a threshold IQ test, even if the threshold is low, is legally suspect. So now indeed many young people who are barely college-ready have to go to some kind of diploma mill college, often at great personal expense, to maximize their likelihood of getting jobs that don't really even require a college education.

That's interesting, I didn't know that about America. I'm in Australia and currently applying for graduate jobs, and every large company I've applied for has made me do one or more pattern completion and/or mathematical and verbal reasoning tests online before even getting to the phone interview stage.

Why is it that management consulting jobs seem to be regarded as some sort of wonder-job that guarantees success? It's certainly a great field and definitely helps, but I think whether or not you can become a management consultant after your stint at an Ivy is hardly the measure of "success" and "impact" that the author of this article connotes.

Here's the straight dope: We need more engineers. Real innovators and builders, who can create real value with their own hands, and not through meetings and Powerpoint Decks.

And if higher SAT scores mean that people can build things that really solve problems and are valuable, then hell yes there is impact. I suspect there's less correlation there between SAT scores and success than whether you got that great job out of college.

The engineering equivalent of Bain or McKinsey is Google. While they don't outright ask for SATs now, they come close. One of the other comments on this post suggests that they used to.

Is this a PR piece created by the company that sells and manages the SAT?

Close:

Gerald M. Bradshaw of Crown Point consults with students on how to gain admission to selective colleges, universities and law schools. Contact him at www.bradshawcollegeconsulting.com or call 663-3041. His email is gerald_ bradshaw@post.harvard.edu.

It's in this guy's best interest to make people think the SATs are important, and getting into elite colleges is important. I put no value in this column.

bah. I didn't take the SAT and I'm a PhD candidate at a major public CS research department. Yeah, not taking that has really impacted my life, let me tell you about it.

But what if you want to get a low-paying entry-level job that will require you to live in India? Then you need high test scores!

(That's what I got out of the article, anyway.)

Did you take the GRE?

Actually, I'm curious if GRE scores have the same impact as SAT scores at the firms that care about standardized test scores. Or the ACT, for that matter.

I took the general GRE; I didn't take the CS GRE, in fact, I was told not to by my advisor. The general GRE is usually a graduate school requirement rather than a department requirement.

My comment apropos Hacker News is that this kind of hiring process might provide a strong incentive to form a start-up and become the boss rather than one more guinea pig for some other company's human resources process.

That was my theory too, all throughout my teens and early 20s. Then I actually did it, and now I'm not so sure.

Forming a startup exposes you to a lot of completely arbitrary randomness - probably more than your average HR department. You think that being denied because of your SATs sucks - how about failing because a competitor you never knew about launched literally 2 days before you and stole the limelight? Or getting turned down from YC for unknown reasons? Or having your startup fall apart because your cofounder gets into business school? Or succeeding because you just happened to end up on SlashDot, or because your chief competitor couldn't deliver a button for the Netscape search bar?

You'll always be a guinea pig, because human beings act in capricious and arbitrary ways. You just have the choice of existing at a customer's whim, an investor's whim, or an employer's whim.

If anyone asked for my SAT score I'd laugh in their face. I don't even remember my score, honestly.

Well, how old are you? If you're in your 40's, that might make sense. If you're 22, not so much.

I'm 25.

I'm 27 and I still remember mine. I even remember which question I got wrong.

What was it? (I just registered for the SAT after reading this article. Did you pay the $12.00 to see the questions you got wrong and info like that? Looked like a ripoff to me. What's it gonna be, a 2-page printout?)

It basically came down to not knowing what "ennui" meant. I think it was something like "apathy : indifference :: " and then I put "boredom : ennui" or something like that, because it seemed plausible and none of the other options really fit. But it was one of the other options. :-( Funny, too, since I'd just read that "apathy, indifference, and ennui" are three of the most common SAT words, and I should've made sure to know what each of them meant. But I was pretty religious about not studying for tests, so I didn't.

I did pay the $12 for detailed score reports, but I knew immediately coming out of the test that I'd flubbed it. I took it the same day as my sister, and I knew I didn't know what "ennui" meant, so I asked her and she'd gotten it right. Was still hoping it'd come in under the freebie limit thanks to recentering, but no: my year, you could get up to 2 wrong and still get a perfect score, but this was #3. (The other 2 were careless mistakes; I don't even remember them.)

Investment banks and consulting firms almost always ask for your SAT scores when I was coming out of college. It's just another filter to "guess" how smart you. It's unfortunate that most firms assume high SAT score = high IQ.

But put yourself in their shoes... if you receive 500+ resumes per school, and you only have time to interview 24 candidates in the first round, and 50% of them have 3.5+ GPA, what do you do to narrow down the interview list?

If I am asked (other than, say, at a standard carbon copied HR job application form -- where I will leave these fields blank) for my SAT score or a GPA, that will be last question I'm asked by that company.

Damn, I didn't take the SAT because I didn't need it for my education path. How old can I be? :-P

I know an IT firm in Shanghai that gives an IQ test to all new employees. You have to score over 140 to stay on past your probation period.

Although IQ and SAT might be good indicators, fortunately, I can actually interview programmers and get to the heart of what I need.

I'm surprised at how long your SAT, but more importantly, your GPA, affect getting a job.

Agreed, I've seen jobs postings and had inquiries, and not entry-level, where a requirement is a 3.5+ GPA. GPA is like IQ in that employers can't ask for it. But on the other hand if you don't volunteer it (and it's 3.5+), they don't have to hire you, either.

Google asks for it:

http://www.google.com/intl/en/jobs/joininggoogle/resume.html

huh; either I was misinformed or, it was a state-specific constraint.

methink mine was in the 1000 range, zomg i'm doomed!

SATs are taken as an approximation for a person's IQ, which is illegal to use as a hiring metric. IQ (and its legal proxies) is a crappy hiring metric, but unfortunately, it's better than pretty much everything else that's used (GPA, interview, resume).

Reply on news.ycombinator.com