Cargo Cults and Interview Questions | Hacker News Reader

Cargo Cults and Interview Questions

raganwald.posterous.com

229 pointsraganwald158 comments

This fable reminded me of my favourite, unfortunately all to real, cargo cult experience.

I gave a client's boss a copy of DeMarco & Lister's "Peopleware" to read. I was younger and more innocent at the time and thought it would persuade him to change some of his incredibly damaging management practices. He'd produced an incredibly adversarial fear-based working environment that was killing productivity.

For those who've read the book (and if you haven't you should... now... seriously....) this guy was the poster-boy for the Teamicide chapter.

He came back to me after the weekend telling me how much he loved the book. Could really see how it could help the organisation. Would be making changes this week. I walked away feeling smug.

What was the lesson he drew from the book? Of all the lessons that he could learn?

In one chapter there is a brief section on how to spot jelled teams where they mention signs like going out for a drink together after work.

So he decided to take the company bowling on Friday nights.

Attendance compulsory.

Correlation. Causation. So easy to confuse.

(At this point I'm less smug.)

Let us say the result of mandating adversarial teams working fifty hours weeks socialise around a competitive game where alcohol was available was not ideal. During week 1 there was shouting. Week 2 resulted in a fight. There wasn't a week 3.

On the bright side it was the final straw the caused some folk to resign :-)

Did this manager spot the causation between compulsory bowling and folks resigning?

It sounds like the manager would see the causation between bowling and resigning and just force a different compulsory activity.

He hates these cans! Stay away from the cans!

http://www.imdb.com/title/tt0079367/quotes?qt0271097

I'm unsure. For some reason he no longer talked to me :-)

As someone sitting idly in a rather crowded Sunnyvale hotel lobby awaiting my own interview, I'm prepared for a Bob, but I'm afraid I'll get a Ted.

I'm not a formally schooled developer. I don't know a heap about traditional computer science. I don't know all the sorting algorithms off hand (but could throw one together with a little time and reference). I'm not even fluent in all the jargon, vernacular and glossary terms that devs bat around.

I don't know if I'm the best for this job, but I know only 3 short years ago, I was doing graphic design and 3 years before that, I was trying to make good on my degree (and dream) of journalism.

None of us choose how we learn, but I think the best of us just know how to.

Good luck! I hope you end up with a Bob over a Ted.

This story was poorly written. I had to re-read it a few times just to understand who was speaking when.

Personally, I'd kill to be able to write this badly. Not a human you understand, but something. Maybe a bush-fly.

I'd hunt down a whole family of bush-flies, and those buggers are prolific!

Had a random interview scheduled with Amazon a couple weeks ago when they were in town, they didn't even have regular whiteboards because of some mixup. So, they put these pieces of plastic on the windows that was adhesive on one side, worst whiteboarding experience so far. I couldn't concentrate because of all the crap going on outside, poor interviewer probably wondered how I tied my shoes in the morning.

Hope you have better luck ;)

I was in one interview at a local startup where there was a whiteboard in the room, but none of the engineers who interviewed me asked me to touch it; they were satisfied to ask questions of the form “what do you know about Java Foobar Technologies?” After an hour or two, I could feel the whiteboard... looming... blank... silently warning me not to take this job.

Wait, so you got an interview that dispensed with the whiteboard BS.. and you thought that was a bad thing?

There is no silver bullet. I see a sort of taxonomy of techniques, the relative quality of which is very dependent on the person interviewing.

An assumption: it is bad for the prospective employer to see no code at all.

I see two basic options:

1) see code written before the interview, and

2) see code written during the interview.

Option (1) has a few sub-options:

a) look at open source work,

b) ask for private personal work to be made available, and

c) ask for custom work of your choosing.

All of these share the weakness of requiring work outside work hours. There are many qualified candidates who don't have work of the (a) and (b) sort available, and while many see option (c) as a welcome intellectual challenge, many others see it as a tedious (or even offensive) waste of their time.

Option (2) also has a few sub-options:

a) provide a computer and a project or set of coding questions to be solved in your absence,

b) same as (a) but with pen and paper,

c) ask a question and watch on a projector or over the shoulder as it is solved on a computer,

d) same as (c) but with pen and paper, and

e) same as (d) but on a whiteboard.

Option (a) is basically the same as (1c) above, but attempts to mitigate the waste-of-time factor by using already-committed interview time to do it, and option (b) is just a much worse variant of the same. The weakness of these options is that it is impossible to get an idea of a candidate's thought process while solving the problem. Option (c) is really good for interviewers but very high-pressure, which selects against many qualified candidates. Option (d) and (e) are identical, but some people hate pen and paper, others hate whiteboards, and most hate both! It is imperative when not using a computer to avoid requiring or judging by the use of things that computers help support, like specific languages, specific APIs, or syntactic accuracy.

Or is this all wrong - have you found the general solution?

Thinking that all interviews which use some form of whiteboard BS are bad is just as much a cargo cult idea as the use of whiteboards.

Personally I don't think that using a whiteboard necessarily means "whiteboard BS". There are good and bad ways to do it. Getting nitpicky over syntax and expecting the candidate to have everything memorized makes for whiteboard BS. Concentrating on overall structure and approach and expecting reasonable questions from the candidate makes for a good whiteboard experience.

Don't worry, you'll be fine :)

Can you write code? Yes? Well then you probably can fit the role of somebody who is hired to write code.

Nobody on the planet is expecting you to waltz in the first day and grok their entire stack and their entire codebase; expect the first couple of months to be just you getting your head around what you'll eventually be doing.

Don't sweat it, you'll be fine :)

This isn't correct in my experience. Interviewers do typically insist on more than simply being able to write code and it is common for them to try to hire someone who groks as much of their stack and codebase as possible from day one.

edit: but maybe it is not the same in Silicon Valley?

Among CS-led technical departments, this does tend to hold in my experience. Like seeks like, and as much lip service is paid to meritocracy the language and motivations of the industry are largely driven by those who have paid their dues to the ivory tower.

I agree with the others: you'll be fine. And if not, feel free to apply to our company :-)

These are the same things I worry about in learning to code on my own -- Will I know the "right" terminology to "prove" to an interviewer I'm capable of doing X task in Y time?

A funny coincidence, i did journalism too -- you're not missing out on ANYTHING (I'm trying to escape the career myself, even contemplating school).

That's part of the benefit of a CS degree - you get a common vocabulary for common problems and techniques.

> I'm not a formally schooled developer. I don't know a <strong>heap</strong> about traditional computer science.

Well you sure do talk the talk. ;)

Ba-dum---Crash!

;) As I wrote it, I was tempted to kick myself.

See you do know the venacular

I don't know how it is with most interviewers, but for interviews that I do: If you don't know a term, just ask. I'm more interested in whether you're familiar with the concept and can apply it than whether you know the appropriate term. Chances are that you're familiar with the concept already and are just not familiar with the name.

If you're in the interview room, I'm not so concerned about book learning and knowing facts, but whether you have the correct amount of knowledge to do the job and the ability to apply it.

The best piece of technical interview advice I've ever gotten: It's much more impressive to NOT know something, ask about it, and then be able to understand it and apply it to your answer during the course of the interview, than it is to just already know it.

agreed, a few weeks ago I was interviewing and the interviewer schooled me on python generators and cooperative threading (pretty much all my knowledge is around erlang and message passing / explicit threading)

I got the job, and learned something along the way

  You can know the name of a bird in all the languages of 
  the world, but when you're finished, you'll know 
  absolutely nothing whatever about the bird... So let's 
  look at the bird and see what it's doing — that's what 
  counts. I learned very early the difference between 
  knowing the name of something and knowing something.

- Richard Feynman https://en.wikiquote.org/wiki/Richard_Feynman and https://www.youtube.com/watch?v=05WS0WN7zMQ

If they don't hire you because you don't know the most recent buzz words you probably don't want to work for them. A good job especially in your cases is one where you will learn and feel challenged. If you really knew everything they wanted the job would get boring.

Hey! If it doesn't go well or you change your mind you can always come back to journalism now. We are all desperate for people who can write good code and most of us aren't formally schooled either.

The symbol of the thing is not the thing: people (should) care about the thing. You don't need a degree to get your head around algorithms. If you can write a simple binary search, for example (arguably the most fundamental algorithm), that's a plus. Therefore a good interviewer will get you to write binary search (the thing) on a whiteboard without saying "binary search" (the symbol of the thing) and see if you can reason your way through it. They will be much more interested in your thought process than whether your syntax is correct.

Perhaps my terminology is off, but as a minor nitpicking point I have never heard of this type of constructor refered to as a copy constructor. A copy constructor is a constructor for type Tthat takes an existing T and initializes the new T to have the same underlying state. So something like

  class Foo
  {
  public:
    Foo(const Foo& other) 
    {
      //copy state from other Foo here  
    }
  };

What this example is talking about is a normal constructor that can be (perhaps unintentionaly) used in the context of an implicit conversion, so something like

  class Foo
  {
  public:
    Foo(int heyThere)
    {
    }
  };

This allows the C++ compiler to create an automatic conversion (in some instances) from int -> Foo simply by inserting a call to this constructor. So it can cause code like this to compile even if it was not intended

  void HandleTheFoo(const Foo& theFoo)
  {
    //Do something here
  }

  void SomeMethod()
  {
    Foo intended; //assume there is a default ctor available even though I didn't show it above

    int whoops = 5;

    HandleTheFoo(whoops);
  }

Marking the constructor explicit prevents the compiler from using it in an implicit conversion context.

I’ve heard it that implicit conversion is a specialization of copy construction. So you can still use explicit Foo (const Foo& other) if you wish to force all construction of Foos to use the new keyword and not to be passed by value. But I’m certainly willing to accept the suggestion that the two are distinct, in which case my understanding is that the explicit keyword blocks implicit conversion (as described in the story and in your example) and I understand that it also blocks implicit copy construction.

Ah, that makes sense. Though technically you don't have to use the new keyword unless you want a heap allocated instance, something like

  HandleTheFoo(Foo(5));

or even

  HandleTheFoo(Foo(otherFoo));

is perfectly cromulent with the constructor being marked explicit. But that is just me engaging in terminological quibbling, I get what you meant/said in spirit and don't disagree that people asking questions they don't have a deep understanding of themselves can only lead to fail.

Thanks for the kind words. And the reminder about stack variables!!!

I wasn’t trying to suggest that Ted didn’t have deep knowledge of C++, as we can see his fictional knowledge is far better than my distant memories. I was trying to suggest that he was engaging in copy-and-paste interviewing, and that it’s a bad idea for all of the same reasons copy-and-paste coding is a bad idea :-)

Right. The term is “conversion constructor”, one that can be called with a single argument and is not declared “explicit”.

Thanks! “Given enough eyeballs, all essay problems are shallow."

So we're doing this thread again? Stock interview questions don't test real knowledge, they test knowledge of stock interview questions.

In my experience, the most accurate interview processes is to ask a candidate to solve a short (but non-trivial) problem in the language of their choosing (or pseudocode). Ideally, candidates could be interviewed by a few people independently to ensure an accurate sample of that interviewee's knowledge. This isn't always possible (due to time and other constraints) but this process better guarantees that a smart candidate isn't tripped up by a very unusual question, or that a weak candidate doesn't memorize common interview answers.

I think the purpose of this post was to examine why people decide to ask these kinds of questions.

Surely the most accurate would be to ask them to solve a problem closely tied to the sort of job they will be doing (doing variations on FizzBuzz under time pressure does give information, but that information may not be entirely relevant to the job).

And the objective should be to establish a level of capability which is acceptable for business purposes, not to try to optimize on intelligence.

Be truthful with yourself about whether you really want the smartest guy possible. The smartest guy sees the boss's (or company's) bullshit and is more likely to say something about it. This makes trouble. The smartest guy may be a "bipolar" type. The smartest guy, with the mind that needs meat to chew, is more liable to implement things in a clever way than one which average joes will be able to maintain and extend easily. You probably don't want the smartest guy because he probably isn't as good on some other dimensions.

What I see in this industry is that we project intelligence onto the people with traits which satisfy our prejudices (typically people with high class/status). Then we filter on this perceived-intelligence with the rationale that we want a company with people who are as smart as possible. The actual effect is at least as much to favor people with high social status and self-promotion ability and physical attractiveness as it is to favor smart people.

Hiring processes are irrational not because nobody can ever work out the right processes, but because we work backwards from unexamined pictures of "the right candidate" to the processes which will select that candidate.

This is pretty much what I've always done - describe the sort of problems they're likely to encounter in the role they're being hired for, perhaps mentioning a few challenges we've had (and possibly already solved), and ask them to explain how they would solve the problem. It's pretty difficult to bullshit your way out of answering a real problem.

Other than adding in some rudimentary skills tests (say, FizzBuzz + walkthrough writing the code with them), and checking if they've developed anything out of their own initiative, I can't see why you'd bother asking anything else.

I really like the theory of this, but in practice I have a hell of a hard time figuring out "real problems" for which the explanation and solution would occupy less than an afternoon.

Spot on. Asking questions about esoterica (defined as any language feature that's >0 levels above their current level of knowledge!) is useless.

I'd rather get a candidate to write some (relatively simple) code using what they do know about the language, and then we can dive into the code and see if they really understand what it does, and what happens if I poke this or prod that. I don't even care what language they use - although if they pick one I don't know it may be harder for them because I'm going to ask pointed questions about how parameters and equality work in that language.

The interview "question" is just a starting point for a conversation. Trick questions and minutia don't serve that purpose at all.

Exactly. It's not if they know the answer that matters, but if they can determine the answer. Poor interviewers only test knowledge, not ability.

Poor interviewers only test knowledge, not ability.

Right. There are lots of very capable people who have optimized by offloading a lot of information from their heads because they can get it from their tools. These are very different from cargo-cult programmers who are only guessing about which completion in their IDE they should use. However, the two types will be unable to answer many of the same questions when away from their IDE.

Perhaps a dumb question, but why would you use an IDE? I'm learning Python (my first language), and have found about the only thing useful to me in IDLE (the IDE for python) is the autotab function.

Am I missing something? Or are IDE's a waste of time? (I usually just use gedit, fyi, on my ubuntu box)

Particularly for Java, where there's a lot of boilerplate that can be autogenerated IDEs are very useful. IDEs are great for method signatures and completion, as well as integrated debugging and unit tests (1 click run the tests again, add in your breakpoint and check your variable).

Some languages lend themselves to IDEs easier than others. For Ruby I use textmate, for Java I use eclipse. For Perl I use vi, and when I did .NET I used Visual Studios.

In the case of Java and Eclipse, and our million LOC codebase, the IDE provides a breathtaking boost in productivity. I find the main benefit to be the realtime compiler which catches compile-time errors, followed by ease of navigation through the codebase, and the remote debugger and code inspection.

I often think that not having this tool would be like what primates' self-awareness must be like: a glimmer of insight, but even the simplest thought requiring impossible effort, as in a thick fog.

IMO, the little things add up fast. Syntax highlighting can make things easier to read. Linking ('s and )'s let's you close a conditional without counting the number ('s less the number of )'s. Syntax aware re-factoring let's you safely change all x's to index etc. Built in language support allows you to catch many syntax errors before you hit compile and the help goes on and on.

PS: And don't forget the big things like built in debugger support.

The main things I like in an IDE are discoverable functionality (e.g. in Visual Studio, variables that changed since the last breakpoint are red--I'm sure gdb has acquired this ability since I first looked at it but it's questionable if I'd've ever gone looking for it on my own) and refactoring (e.g. changing the name/signature/return type of a method throughout your codebase). IDEs are typically big wins when you're writing in statically typed languages (especially ones without a REPL) and less so in dynamic ones - in Python I'd be perfectly happy in Notepad for at least 75% of my programming because I do so much exploratory programming in the REPL.

The Refactoring Browser is a huge win in Smalltalk.

Easier debugging, faster code navigation, better refactoring tools, auto complete, inline documentation, tighter integration of unit tests, and probably many other features I'm not thinking of. Sure I write one off python/ruby/ps scripts in notepad++, but for me any project of real size and complexity will receive a huge productivity boost from a well configured IDE.

Your mistake here is thinking that IDLE is "the" IDE for python. It is really nothing like most real IDEs, and more of a toy.

As some others said, the language makes a big difference. When I write Ruby/Rails code on my Mac, TextMate and a terminal window are all I need. When I'm at my day job writing C# (a static typed + compiled language), a setup like this would be incredibly painful. Just the loss of the powerful Visual Studio debugger would send me home crying, not to mention having to worry about setting up complex dependancies in a build script when calling the compiler over the command line.

I don't think esoterica is entirely useless in an interviewing context. I've interviewed a few people for language centric positions before and one thing I liked to ask was, "What feature or piece of this language do you find interesting or frustrating?" Someone who's worked a lot with a particular technology is bound to have something meaningful to say. I suppose they could just not be particularly thoughtful about their tools, but I like working with thoughtful people.

The problem is with requiring knowledge about a certain set of "esoterica".

Your question is fine: it's open-ended, the interviewees can answer whatever they feel like (and you can assess their knowledge of the language to some degree), and ideally you get a nice little discussion going, that also covers other topics.

On the other hand "What's up with that explicit keyword and constructors in c++?" is a pop quiz questions.

I agree ... as long as you don't use said esoterica as an immediate qualifier/disqualifier, it can give you an interesting insight the depth of knowledge a candidate has about a particular language. That being said, there are people who know crazy minutae but get into a team and simply can't turn out code at the rate you need them too, for whatever reason. Definitely have to watch for that and pay more attention to how they actually construct and write code.

Agreed. I've met some "language lawyers" that know the standards and libraries by heart, but can't seem to combine them to solve problems. Or they can, but they create overblown space-shuttle architectures that solve a metaproblem that far exceeds the actual problem at hand.

I've found that I don't think about languages that way. I think of things more like "I want to achieve goal X; what are the interesting or frustrating aspects with respect to that goal?"

For example, someone asked me once about the warts in Python. I started thinking about complaints I've heard others say, like Python 2.x's "print >>fileobj" syntax. I understand the reasons for and against it, but I nearly only use print statements for debugging, so it's not one that I want to bring up. Or there's the lack of block syntax, which Smalltalk/Ruby developers love, but Python has different ways to handle most of the underlying requirements.

So there I'm thinking about the different aspect of Python, trying to figure out which ones I want to bring up, but I hadn't fully considered the other issues with the language, in order to figure out which is the most relevant, but then, I don't know how to rank the relevancy because the question was structured without any rank preference.

Now if you had asked "... when used for systems programming ..." or "... as a programming language for scientists ..." or "... compared with Perl ..." then I would find it much easier to give the thoughtful answer you're looking for.

I had the same problem a couple of years back at our monthly Python meetings. We did a code refactoring exercise: given code, make it better. I had a really tough time because I couldn't figure out what "better" meant. The exercise code worked, although it mixed business logic, database interface, and presentation into a single file. But without an idea of the direction the code was supposed to go for the future, I couldn't figure out why anyone would want to modify working code.

Okay, so you go on my no-hire list because you paralyze yourself with questions :P

That's okay - I've been a self-employed consultant for 15 years, since that way I can ask people "what's your goal?" and get away with it. ;)

This was an interesting story. Sounds like an astute interviewer, the kind who can really get at the core of technical knowledge. And of course, people have every right to set whatever standard they like in their hiring.

But every time I read about someone like Alice, it does cause me to reflect on the "hiring troubles" companies are experiencing, the "shortage" of top candidates.

She has contributed to open source, shown the ability to learn and write small apps in new languages. She can code, definitely, but she shows some weakness... NO HIRE. Not only no hire, she's a faker, a cargo cult-er, she "pulled the wool over your eyes." She's not just unqualified (by high standards, in my opinion), she's insincere.

Please, understand that I'm not saying that hiring standards shouldn't be high, or that weak team members don't do more damage than good. I get all these things. But the bar is high enough that it should be utterly unsurprising that there's a shortage of "qualified" people, and that a lot of people (who might have become good programmers) bail out, get forced out, or want nothing to do with it after their first taste.

I'm ok with these high standards, but I think we celebrate it a little too much.

EDIT: raganwald, jbooth, peter...

thanks for replying rather than just downmodding - unfortunately (for me), yeah, you're right, I misread this.

I'd like to take it down, but once someone has responded, I leave it up with an EDIT...

Maybe I'm misinterpreting both the original post and this response, but it seems that you think Alice is the "cargo cult-er". I interpreted raganwald's post as critical of _Ted_, criticizing _him_ as a "cargo cult-er".

I may have done a poor job of conveying why the story is related to Cargo Cult religions.

The term cargo cult programmer may also apply when an unskilled or novice computer programmer (or one not experienced with the problem at hand) copies some program code from one place and pastes it into another place, with little or no understanding of how the code works, or whether it is required in its new position.

https://en.wikipedia.org/wiki/Cargo_cult_programming

I don’t know if this term can be applied to Alice, but I am pretty certain that the story describes Ted blindly copying the surface form of something Bob did in an interview with Carol into Ted’s interview with Alice with little or no understanding of how Bob's conversation worked, or whether it was required in Ted’s conversation with Alice.

The message was clear, some people just see what they want to though.

I think the article was agreeing with you. The cargo culter was the one who saw someone ask an improvised question about implicit and turns it into a litmus test.

I agree with the story. To interview, you don't turn little details into big questions. You turn big questions into little details.

Interviews are lossy compression of a career, experience, and brain shape, over a noisy channel of whiteboards and human speech. The best you can hope is that your interviewer will recognize the larger pattern.

BUT. There is limited time in an interview, so we try to scry nuances like a soothsayer with a pile of sheep guts. Every detail gets magnified.

Examples:

"You chose to return an error code instead of throwing an exception." Is this an optimization, or have you spent a lot of time on system programming?

"I see a couple of fenceposting errors." Is it nerves or do you not have the right shape of the problem in your head?

"You passed some bookkeeping information in a recursive call that should take care of itself." Do you not understand recursion, or are you misremembering something cool about accumulators and tail recursion?

"You chose to use inheritance instead of composition." Have you ever had to extend or maintain such a design, or did you read about it in a book?

"You used template metaprogramming." Are you a misunderstood genius or do you delight in writing unmaintainable constructs from the dark corners of C++ into production?

"You chose to use Perl but don't know what a hash is." Do you have any questions for me?

>"You chose to use Perl but don't know what a hash is." Do you have any questions for me?

This reminds me of someone who once told me they knew C and were proficient at it, but they just "didn't get pointers and I don't use them."

Scoff if you like, but I hired that guy. He's a fellow at Adobe now. They bought his startup.

We interviewed a Busuness school Finance senior to help on our COMSTOCK satellite feed parsing project.'Do you know C++?" we asked. "Yes" he lied, then "Can I come in over the weekend to get started?" How enthusiastic! Sure!

Turns out he spent the weekend studying code, learned to program C++. I didn't see him for a few weeks, then consulting on his code over something I mentioned "This would be better allocated". "Yeah, I wondered what the pointer stuff was about". I was astonished he'd come so far on guts and smarts. I explained malloc, pointers, he was off again.

Put another way, you had (have?) astonishingly poor interview techniques but this one time you got lucky?

You could pretend that. Or you could say, we valued his intelligence and readily-apparent ability to learn over 'fizz-buzz' nonsense questions and cargo-cult interview style.

Looking back, were they overqualified or underqualified? If you had tested the second guy's C++ skill (sounds like he couldn't have done Fizzbuzz in your interview), would you have disqualified him? If you didn't think C++ was that important, why ask in the interview?

All I'm trying to get at is if you take your experience as a rule.

The population of people who lie about their programming skills is unfortunately all too high. The subset who can make up the difference in two days, unfortunately all too small. I think a Google or Amazon has to play the percentages to keep a high bar. The cost of a bad hire reverberates through the organization.

Which is why you should learn more about a candidate than their current programming toolset. Like their adaptability, their insight into problems and their convolutions. Not about their facility with the current fad language.

How do you strike a fair balance between ridiculous interview questions (such as the example here, write code interfacing with some library from memory) and actually finding out a candidate's chops (assuming they don't have an open source history / github account). I think requiring candidates to have contributed to open source to be an unacceptable solution (although I can certainly see lending some weight to candidates with such credentials)

Anecdotally: We had a guy in for interviews a few months ago. In the phone interview prep by HR, it was mentioned that there would be some written “tests,” and the guy was extremely resistant. We had him in anyway and he did very poorly. The test was pretty simple: some stuff about magic methods in php, a fizzbuzz type question, and a few jquery selector questions (it was for a front- and back-end position), etc. Were we to granular with the questions? Did we fall into the same trap as Ted here?

1) You do what Bob did; have them write code and then ask why they did it that way.

2) Were the written questions on paper or a whiteboard? If so, you should at least give them the option of doing it on a computer with a screen editor (you know the way that almost everyone has written code since the 80s). We have a few computers with just about every editor known to man (well win32 and *nix only, but we could probably scrounge up a powerbook if we needed to) installed on them along with gobs of syntax-highlighting themes so that candidates can feel comfortable when writing code.

Ultimately some good programmers have trouble writing code on paper, and most good programmers have trouble writing code on a whiteboard (though they should at least be able to sketch out an algorithm graphically or with pseudo code as that's a useful communication tool).

I agree with #2 a lot, although when I start interviewing not sure that would make the first couple of interviews go smoothly. Usually takes me 3 or 4 interviews to get warmed up to the type of problems people like to solve in whiteboard interviews.

When I just randomly interview a single time, I feel like a dolt, do pretty miserably. After a few interviews all the whiteboarding starts to take effect and I do much, much better.

When I interview people I usually explain a problem I'm working on and see if they can think through it, then ask a lot of questions about what they've learned from their past failures. People that can go into detail about something they failed at and learned from usually impress me, as I suspect they won't make the same mistakes over and over. Also, if you haven't failed spectacularly with something very hard and you've been a software dev for 10+ years, you might not be doing it right.

Obviously, I've been a dev for 10+ years, and I can't pass whiteboard tests very easily, so I suppose maybe I'm not doing it right, either...

Simple tests (what is this jquery doing? why is this ruby script failing?) are great as bogofilters as long as you actually use jquery and ruby on the job.

I interviewed somewhere a few months ago and had a former programmer turned product manager grill me about b-trees (this was a Ruby on Rails position). I was not surprised to hear later that he was the one who gave me the final thumbs-down, I assume he thought I wasn't worth it because I didn't work at the same level of abstraction that he did when he was a coder.

He was seeing if you had a depth of understanding. I do that too.

I want to see that people are aware of what's going on below them. Not that it's just "magic" to some keyboard basher. ruby on rails attracts an extreme number of keyboard bashers, so they get extra scrutiny.

That's very different from grilling someone about b-trees, though. B-trees are fairly irrelevant to Rails development; it'd be like grilling the interviewee on assembly.

That's not to say that you don't need to weed out the people who only know how to recite various incantations to make magic happen; a knowledge of what's actually happening is very important. But at this level of abstraction, the correct focus should be on different, more relevant things: HTTP, caching, security, etc, which Rails (I'm guessing) handles for you but you should still know.

I get that, but when the interviewee is not just a "keyboard basher" then it suggests that the interviewer is the one lacking depth of understanding as they are conflating superficial ignorance with fundamental incompetence.

It makes it worse that you just know that guy stayed up the night before studying b-trees for hours just to show you up.

One option would be to ask them to bring in some code they've written, and then talk through it with them in the interview. Even simple stuff like greasemonkey scripts could be pretty informative as to the candidate's level.

I'd be very wary of asking questions about potentially-obscure corners of tools (I don't know how obscure magic methods are in PHP) -- in a real work environment, you could look it up in a moment, but you usually can't in an interview. You could easily trip up a programmer who thrives with the references to hand, but isn't yet confident enough with the tools not to struggle without the references.

If I were to ask 2 questions, I would:

1) Ask something simple, like FizzBuzz. You'd be shocked at how many people talk a good talk but can't actually write basic code.

2) Ask an open-ended question, like "how would you design a multi-player chess game". That leads to a discussion that lets you assess all sorts of traits, the most important of which is communication style.

So "ability to write basic code" and "communication style" are your top two priorities?

These two are "must have".

Yes, in the same way that "breathing" and "drinking water" are the top two priorities for my life.

By that analogy, it seems odd to hire someone only to find out they can't eat, and then a month later you need to hire someone again.

It's sort of more like embarking on a months-long nuclear submarine journey only to discover that nobody packed any food.

In this analogy, you are the company hiring programmers, and the things you need to survive are that your programmers can write basic code, can communicate effectively, and some other things — probably "can design software" or "doesn't rathole excessively" is the next item or two on the list. You're still in a lot of trouble if your submarine doesn't have any food onboard, but the situation is not nearly as bad as if the submarine lacks means for producing air and drinking water.

I think you were fair. jQuery selectors are pretty easy stuff, and you can pretty much guess at the jQuery interface and probably be right.

This is coming from a python guy who does very little javascript.

If he did get some of the methods wrong you could always ask the candidate what he thought the method did. If he knew there is a method to accomplish a specific task and he just got the name of it wrong that I'd say he is good to go. He can always google that kind of thing on the job.

I had never heard of the fizzbuzz question, so I googled it. This is what I found: http://www.codinghorror.com/blog/2007/02/why-cant-programmer...

..... I sincerely hope he's kidding. Having less than 8 months experience with a language (AT ALL!) I can write code out on paper that works. IS this 95%+ figure correct ? :S It's come up a couple times in google results, but I dont see where people are getting the number.

If you write it on paper, are you sure it's correct?

Not really related but I wonder about this all the time. Most of well known companies (Microsoft, Amazon, Yahoo, Google, Facebook etc) have a lot of data structures and sorting algorithm related problems as part of their interviews. They might require you to explain the algorithm or write code for a given data structure problem. Do these companies use data structures like binary trees (with all the variations) and sorting algorithms a lot ?

I'm asking this because, the companies I work for are mostly tier-2 companies which hardly use any of this stuff and generally get away by using proper libraries. So when ever I have to prepare for attending interviews of the tier-1 companies, I'm forced to re-learn most of this stuff - because knowledge fades when you don't use it.

I used to work for Amazon, and used to interview a lot while I was there.

The answer is: it varies, but generally no. There are some roles where you're going to be spinning mad data structures & algorithms day in and day out... but for the most part you write code that isn't overly dissimilar from what's written everywhere else.

This is my own observation and hypothesis: "tier 1" companies have very algorithm and data structures heavy interviews because their signal to noise ratio is horrific. Being household names, you have tons of people applying who can't even Fizzbuzz. In this environment the natural reaction is to check, first and foremost, for CS fundamentals, because you can't assume that it exists and is just rusty.

I now work in startup-land, where hiring is so much easier, since people who find their way to your doorstep are overall much more qualified by default.

I'll admit, when I interviewed for Amazon, I always hit with algorithms and data structures questions first, because by my own unscientific estimates, at least 50% of applicants at the phone interview stage do not even understand what a tree is, much less how they might go about crawling one or representing it in data. I'd reckon about 10-15% cannot pass Fizzbuzz.

No decent company should allow a single interviewer to shut down a candidate. I am a regular interviewer (phone and in-person) and I've got a few pet problems that I throw at all candidates who pass the simple introduction. I would never can someone on and 1 type of question, but I think it's fair to say that missing 2 or more conceptual questions will pretty much end the interview for me. That's why we have at least 2 2-person teams interview in-house -- if they passed the phone screen and make it in our doors they deserve it.

Great example and a great follow up to "I don't hire unlucky people" As always a pleasure to read your blog.

I entirely agree with raganwald's point point: language trivia is useless. I'd go further and say if someone knows actually knows C, and has a grasp of using OO in a way that's not painful for others working with them (prefer composition over inheritance, etc...), they can learn C++ on the job. If they know all C++ trivia, but lack a strong knowledge of either C, or good OO, they can do harm to a code base.

However, this isn't to distract from importance of using explicit and copy assignment. Here's a public service announcement: really, if your C++ class has a constructor that takes one argument, please use explicit.

Additionally, please disable copy constructor and assignment operator, unless you actually implement them yourself. Here's a free macro for you, courtesy of Google's C++ style guide:

  #define DISALLOW_COPY_AND_ASSIGN(TypeName) \
    TypeName(const TypeName&);               \
    void operator=(const TypeName&)

  class MyClass {

  private:
    DISALLOW_COPY_AND_ASSIGN(MyClass);
  };

I will thank you for this.

Errr... if you concluded that "language trivia is useless" was the point, I think you fell into the actual trap the essay was trying to warn against. For example, broad knowledge of language trivia would be a positive attribute when hiring someone to design a language.

The point, I think, is that asking interview questions without understanding why the questions are appropriate is worse than useless - it's actively destructive to the end goal of hiring good people.

You can generalize this Cargo Cult behavior to any field - blindly carrying out activity without understanding deeper principles is likely to produce undesirable results.

boost has an elegant solution that I like, inherit from boost::noncopyable.

  class Foo : boost::noncopyable
  {
    // ...
  };

I doubt anyone knows all C++ trivia, and that if someone even knew most of it, they could be that clueless about OO design.

I work on libraries more than apps. When I want to hire someone to work with me on libraries, you better believe I'm going to look for a C++ expert. And the explicit keyword is crucial in a lot of cases -- sometimes it can mean the difference between code compiling or not, but even worse is the bug where a parameter is missing and an int is converted to be some class it shouldn't be.

Using existing libraries and just assembling bits into apps? Sure, learning on the job can be OK (though honestly I wouldn't trust C++ written by anyone using it for less than a year full time). Though even then if you don't understand C++ well you're likely to shoot yourself in the foot with the language.

But there are some times that I really, really want someone to know the explicit keyword, at a minimum. Your example of DISALLOW_COPY_AND_ASSIGN is also good to know -- and it's likely in the Google style guide so that people don't have to be experts.

But I'd rather have people know why it's important and how it works than people who are just trusting to the compiler and luck to make everything work right.

I've seen C programmers clear a class that contained a std::string with memset(&foo,0,sizeof(foo)), which happened to work on one implementation but failed badly on another. I've seen people take the raw pointer out of a smart pointer and attempt to manage it themselves -- including trying to wrap ANOTHER smart pointer container around it later. I've seen people fight with a bug for weeks because they didn't bother to initialize all their class members (fixed that bug in about 10 minutes on code I'd never seen before).

The list goes on, and the list of "coding standards" we created to try to enable people who weren't good enough at C++ to work effectively with our C++ library got to be so large that no one could remember it all, rendering it useless. [1]

So yes, if I'm going to have someone work on C++ for me, I either want them to have at least a solid year of real experience, or I'm going to want to code review every line before it gets committed to main. Maybe I'll still hire them, but I've seen so many examples of people misusing C++ (and then having to fix their bugs later) that I've been more than a little burned by it.

Even more esoteric than explicit is the mutable keyword. Major bonus points for anyone I'm interviewing to know what that one means and when it's used. But no major negative if they don't know -- it has very few compelling use cases, after all, so I can imagine people not remembering it.

[1] https://developer.playfirst.com/standards

Huh, your comment makes me wonder if I should look for contract work doing C++ development. I haven't done it seriously since the last millennium, and basically I've learned to program since then [0], but I know all the stuff you're talking about cold. (Except mutable, which took me a few seconds to remember, and then I looked it up to be sure.)

It's just that I feel like higher-level languages like Ruby, Python, or JS are so much more productive for most things, most of the time, despite things like Rachelbythebay's interesting set of posts about hacking together stuff in C++ [1].

But, aside from things like that, I guess there are some domains where C++ is a viable option and Lua or whatever still isn't. And it sounds to me like the average quality of C++ has gone up since I was doing it, to the point that most people have read Alexandrescu and never call delete explicitly, so it might not be quite so painful?

Should I be taking advantage of being one of the Fat Guys Who Know C++? [2]

[0]: http://lists.canonical.org/pipermail/kragen-tol/2007-March/0... [1]: http://rachelbythebay.com/w/2012/01/14/blort/ [2]: http://medriscoll.com/post/9117396231/the-guild-of-silicon-v...

Great links. I've certainly made a habit of hanging out with those Fat Guys, though at this point I'm in Boulder, so while they still have neck beards here, they tend to be pretty thin. Though honestly we stopped playing Settlers years ago, and have moved on to games like San Juan and Dominion. ;)

To tell the truth, I have a love/hate relationship with C++. Yes, I know it really well at this point. I've learned everything I know because it (at the time) was the only tool reasonable to get the job done. Now I actually avoid using it as much as possible.

The perfect combo for me (in games) is C++ for performant code combined with tight integration with Lua. I hear you on higher-level languages being more productive, which is why most of my game code ends up written in Lua. (I'm far from alone in using this combo -- most major iOS games seem to use the same combo, as do a lot of games in general.)

Why not use one of the three you mentioned? Size and speed both.

LuaJIT is pretty much the fastest dynamic language around, and the overhead of linking into a program is tiny compared to Ruby, Python, or JavaScript. The language itself gives you advanced features like coroutines with upvalues and operator overrides, but at the same time it's super-simple -- to the point where non-programmers can use it to script behaviors.

If you're doing server code you can afford to have things be a bit slower and/or huge (relatively speaking). If you're doing games downloaded OTA on a phone, and you've got a 20Mb limit before they have to use WiFi...well, size is critical (an extra megabyte or two means a lot of assets won't fit any more) and phones are far more processor-bound than most server tasks you'd use a scripting language for. On the other hand, with distributions like OpenResty [1] that hand you an extremely fast server and LuaJIT all bound up together, allowing me to create a cheap server that can handle 3000+ requests per second without barely breaking a sweat...well, since I know Lua ANYWAY, I use it on the server as well.

YMMV. :)

Addressing what you asked: C++ development is in reasonably high demand compared to the number of Fat Guys, so it tends to pay well. Having experience in a niche can also be handy -- by virtue of knowing Android C++/NDK development I've had random people contacting me to ask if I want to work for them, with no real promotion on my part. I don't even want to say what my rate was publicly, though I will say that I expected him to turn me down when I said how much I wanted.

Android C++/NDK by itself counts as a niche? That's kind of surprising! I would have thought a profitable niche would have to be a bit more specialized by now... I guess the NDK hasn't been out all that long.

I imagine gzipped Lua bytecode would be quite a bit more compact than gzipped compiled C++; does it work out that way in real life?

Does OpenResty work well with LuaJIT?

[1]: http://openresty.org/

OpenResty ships with LuaJIT 2.0 beta. You have to add a ./configure parameter to get it to use it, but it works well, yes.

As to bytecode...I think it probably does end up more compact than compiled C++, and it seems like it would have properties that would allow it to compress more tightly, but I don't have any numbers on that.

I tend to strip the source and compress and encrypt it; I could convert it to bytecode first, but the last time I tried that it ended up bigger than the compressed text.

>Android C++/NDK by itself counts as a niche? That's kind of surprising! I would have thought a profitable niche would have to be a bit more specialized by now... I guess the NDK hasn't been out all that long.

A lot of "Android" developers are actually Java developers who are trying to cash in on mobile. Serious game developers almost all use the NDK, but serious game developers are in high demand (myself included). So yes, NDK is itself a niche, in that the number of jobs seems to exceed the number of developers available.

On the other hand, by being able to do games in NDK I'm even better off, but just being good at NDK development is pretty powerful. Keep in mind that you also have to deal with Java and the Android stack -- you can't do everything in the NDK. So you need to be someone good at C++ who also can get around in Java.

That's really fascinating information, thanks! Who's a "serious game developer"? I'd think that things like Angry Birds and Bubble Bobble would work just about as well in Jython as in C++, but I guess they're not "serious"?

Angry Birds uses C++ and Lua. Haven't looked at Bubble Bobble.

Not saying you COULDN'T write games in other languages. MineCraft is written in Java, for instance. But my statement still predominantly holds true: Most serious game developers use C++, and many use Lua.

For some definitions of serious, anyway. I guess I mean "ones that have been doing it for any length of time professionally." Or maybe I just mean "almost every one I know." Something like that anyway.

Some perennial favorite "interview questions" of mine:

1. Tell me the difference between final, finally and finalize in Java. (Of course it was completely irrelevant to the discussion that my resume at that time never claimed that I knew Java; The only mention of Java was a library that I had used as a black box)

2. This is a fairly new one but most software engineers must have seen equivalents over the years :).

Rate yourself on a scale of 1-10 on the following:

-- Machine Learning

-- Convex Optimization

-- Linear Algebra

You know what was really disappointing? The second question was asked to me first by the recruiter (whom I don't blame for asking me the question) and then again by an engineer (after we had just concluded a long discussion on my projects touching upon those topics).

I generally believe interview questions aren't a terrible way to screen people. There are, however, stupid questions and good ones. Ones that test how people think or how good they are at designing/accomplishing tasks are good. Ones that test stupid syntactic knowledge or language specific things are dumb.

How about.. The interviewee gets to know what languages, libraries, components of the stack you are using a week in advance. They then can implement whatever they like using those technologies. Then, they present what they made to a technical member who can ask them questions which would show they have developed familiarity with at least some parts of what they use. The more specific the position, the more in depth the questions.. The interviewee can also ask questions back to the interviewer from a more level position.

In this field, this will probably get you lower quality applicants or kids right out of school. High quality applicants have recruiters banging down their doors, and there are always several other options on the table that won't waste a week of their time.

The point, as always, is that intelligent people find other intelligent people to hire by asking questions that aren't uniform or one-size-fits-all, or which only require regurgitation of facts, but rather by probing how and why a candidate thinks a certain way.

This should be obvious by now. Standard CV reviews, colleges and references say nothing about how efficiently a candidate is able to think through new problems or how elegant their solutions will be. Nor does that mean that formal training and real-world skill exist in an inverse relationship as a lot of self-taught coders (myself included) like to pretend it does. They simply aren't co-related, at least not as far as we can tell from most leading posts about hiring on HN.

So if the goal is efficiency, why not just hire people based on their IQ? What else is there that qualifies people for positions where their bosses 2 levels up have no idea how they're actually executing the work that's put in front of them, and rely completely on them to come to the best logical solutions to new problems creatively, quickly and every single time? The only other standard you might want to look for is trustworthiness, but you can't measure that in an interview.

Anyway, it doesn't matter if someone with 140+ IQ has ever coded before. It's probably worth paying to train them. So let's just cut to the chase. You need people with brains and integrity, and the rest of it is just huff and puff nonsense from HR people who don't have a clue, and couldn't do the job their hirees do.

firstly, there's legal issues with IQ tests in job interviews in the US at least

secondly, IQ testing covers a number of areas, many of which are not necessarily related to the kind of thinking needed for programming. i think most of the verbal stuff in IQ tests wouldn't relate that much

thirdly, even if someone aces the IQ test, there's still no indication that they would actually enjoy programming if they've never done it before. It's kind of pointless to train someone to code if they're just going to get sick of it and quit in 6 months.

and fourth, an IQ test doesn't say anything about a person's ability to work with other people or prioritize things properly. an IQ test provides no protection against ending up with a prima donna who obsesses over brace style at the expense of getting useful things done.

really, i think the best way to find out if someone can code is to look at something they've coded

Is it actually illegal to hire in the US based on IQ? I mean how is that discriminatory more than any other test for employment? Race, religion, political affiliation, disability are obviously all irrelevant in an IQ test... so in a way, isn't it more equitable?

You make a good point about primadonnas and other potentially toxic personality types with high IQs, but you can screen that too. And I mean, I don't care about IQ. To me, pretty much anybody with talent should be able to come in with something awesome and say "look what I did", and if it's great, then who cares? Hire them. Obviously this isn't an argument for creating a new narrow-minded structure to replace the current one; I'm just talking about prioritizing along the essential lines instead of dancing around the real question. Isn't someone who codes and who can give you a fibonacci sequence in realtime more valuable than someone who codes, but can't? I can't do it, but I'd hire that guy.

Race, religion, political association, and the like are all socioeconomic factors which have been shown to have an effect on an individual's score on an IQ test.

The classic example goes as follows:

The testee is shown four pictures: a teapot; a saucer; a table, and a water pump. They are then shown a teacup, and asked to match it with the appropriate picture. The upper-class testee, for whom tea was always served in a cup on a saucer, selects the saucer. The middle-class testee selects the teapot, as the tea obviously goes in the cup. The child in sub-Saharan Africa recognizes the teacup as a cup, a so selects the water pump, as water goes in cups.

This is obviously a contrived example, but it shows how the biases of the test-makers can conflict with those of the test-takers, leading those from different backgrounds to score worse on the test.

A real example I heard of a few years ago was with children putting picture story panels in order.

The "correct sequence" was: Get Up, Eat Breakfast, Go To School while many of these children were getting subsidized breakfasts at school.

I think anyone with an IQ over 80 would agree that's more of a red herring than a test of actual intelligence, so, if we can dispense with absurd fringe examples of the abuse of the concept of "IQ" maybe we can get down to the nuts of what companies are looking for, what programmers are trying to provide, and put a stop to this dramatic yarn about "how hard it is to find good help" once and for all.

The example was for small children and speaks more to the fact that IQ tests are in practice more discriminatory than they are designed to be. That's all.

If you can come up with a way to test for general intelligence without inadvertently biasing your tool in favor of a specific ethnicity, social class, or gender we're all ears.

http://en.wikipedia.org/wiki/Ravens_Progressive_Matrices

Curiously, American minorities score more poorly on culture-neutral tests like Raven's Matrices. They do better on culturally biased tests because they can pick up mainstream culture knowledge from TV and movies.

Couldn't you argue that race, religion and/or political association and the like have an affect on programming tests also?

>Race, religion, political affiliation, disability are obviously all irrelevant in an IQ test... so in a way, isn't it more equitable?

Except that cultural background is not, unfortunately. And it's highly correlated with the first and probably weakly linked to the second and third classes as well.

Let's assume a test that isn't culturally biased. The idea behind the test should be to determine the raw underlying intelligence of a person; culture shouldn't play into it at all. The fact that some IQ tests seem to discriminate - referring to the legal explanations above - appears to be a flaw in the tests given in those cases. If a test is biased toward a race or culture, then it's not an accurate measure of intelligence; we can't even call it an IQ test. It doesn't meet the definition of what most people are talking about or referring to when they say "IQ test" -- since "IQ" by definition is measured on a sliding scale in reference to the entire population.

For argument's sake let's assume a test that doesn't discriminate or give advantage to any given race or culture. How is that different from a coding test, other than that it requires more general interest and knowledge, and doesn't overly reward memorization of algorithms?

This is a substantial enough topic that Gould wrote a whole book about it, The Mismeasure of Man.

But directly relevent:

* even if the concept of some sort of underlying general intelligence were valid, we have yet to come up with an unbiased test of it.

* And then if you could somehow prove an IQ test was unbiased and germane to the job, then the court ruling referenced elsewhere in this thread wouldn't prohibit it's use.

So you can't prove anything about anyone. No test can qualify or disqualify anyone. What about writing code? Isn't that a kind of basic test of intelligence? Or do you hold that anyone can write as well as anyone else, and there's no fundamental logic or critical thinking needed as long as they're properly trained? If that's true, why all this bother about who to hire -- can't we just hire anybody, and find them equally useful or useless in their jobs?

You've gotta measure people somehow in order to hire them, right? You can spend all day talking about how unfair any possible test is, but the world will continue choosing people based on a multitude of criteria, most of which boil down to intelligence, and all I'm suggesting is that there are more efficient and less circumspect ways of doing that.

Right, and I said if you could create such a test there wouldn't be objections to it being used. But it is hardly a solved problem! The especial sticking point is actually showing that some general test really correlates with how well you do the specific job.

(Go and read the wikipedia article about the relevent court case: http://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.)

Yes, and that book would be a wonderful example in a class on polemic because any relationship to the truth is incidental to its purpose. It ignores the modern science of psychometrics and the vast majority of the material it deals with predates the 60s. Outside of the areas where Gould was an expert ([snail] paleontology) I would actively avoid reading anything by him. If its valuable, someone else would have picked it up, but if it's directly from Gould it could be a rehash of something someone else said better more than a decade earlier that he never cited.[0] Please note that in that case I'm assuming incompetent literature review, not plagiarism. But we also have what looks like another example of politically motivated incompetence or fraud[1]. For specific criticism of Mismeasure of Man look here[2]. For criticism of Gould's general importance as an evolutionary theorist, I quite like this

I am not sure how well this is known. I have tried, in preparation for this talk, to read some evolutionary economics, and was particularly curious about what biologists people reference. What I encountered were quite a few references to Stephen Jay Gould, hardly any to other evolutionary theorists. Now it is not very hard to find out, if you spend a little while reading in evolution, that Gould is the John Kenneth Galbraith of his subject. That is, he is a wonderful writer who is bevolved by literary intellectuals and lionized by the media because he does not use algebra or difficult jargon. Unfortunately, it appears that he avoids these sins not because he has transcended his colleagues but because he does does not seem to understand what they have to say; and his own descriptions of what the field is about - not just the answers, but even the questions - are consistently misleading. His impressive literary and historical erudition makes his work seem profound to most readers, but informed readers eventually conclude that there's no there there. (And yes, there is some resentment of his fame: in the field the unjustly famous theory of "punctuated equilibrium", in which Gould and Niles Eldredge asserted that evolution proceeds not steadily but in short bursts of rapid change, is known as "evolution by jerks").[3]

[0]http://lesswrong.com/lw/kv/beware_of_stephen_j_gould/

[1]http://www.wired.com/wiredscience/2011/06/gould-morton-revis...

[2]http://en.wikipedia.org/wiki/The_Mismeasure_of_Man#Criticism...

[3]http://www.pkarchive.org/theory/evolute.html

I was just using that book as an example of how complicated the topic was!

The problem is that, yes from first principals, a biased test is not a true "measurement of intelligence test", however loads of tests that people have been calling "IQ tests", or are found in books called "Big Book of IQ Tests" or "Test your IQ" are biased. So the term has changed, and includes all these biased tests. If you, as a employer tried to do IQ tests, you would probably find it quite hard to find one that you could be sure wasn't biased.

The Supreme Court has held that a particular IQ test had a disparate impact on black applicants. This makes IQ tests essentially radioactive for most corporations in the US.

http://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co.

Unless the job is in law enforcement.

1. That case was a social justice witch hunt. The purpose was to prove the political piety of the plaintiff.

2. The microscopic number of black engineers have already run the gamut of cognitive testing. An IQ test would not have a "disparate impact". This is why big corporations ask for university degrees: the degree itself is an IQ test.

3. FizzBuzz will screen out virtually the entire African race, and is provably related to actual job performance. After that an IQ test will not appreciably change the statistics.

As multi-racial fellow with an i.q. of 148 with out a university degree making six figures doing software development i'd like to say that I find #3 questionable.

"IQ test" is too broad a term to say if it's "illegal to hire in the US based on IQ." Some purported IQ tests are provably biased towards a particular race. Those are illegal. The rest are presumably not illegal, but if you were sued, there'd probably be a pretty large burden in proving that every IQ question you'd ever asked was both not racially biased and directly relevant to the job opening. Hence, very few business even look twice at IQ testing anymore - it's too deep a minefield.

For more background about IQ testing for employment in America, check out https://en.wikipedia.org/wiki/Griggs_v._Duke_Power_Co. .

Interestingly, rejecting candidates because they score too well on IQ tests seems to be totally kosher in the US.

Who would do such a thing? Cops.

I pray this is a snide swipe at the police and not actually true.

Nope.

http://abcnews.go.com/US/court-oks-barring-high-iqs-cops/sto...

I used to work in a place in LA with a regular group of cops who came in every night, and one guy told me exactly the same thing. It certainly explains a lot about cops in America, and the US in general. You can reason with most people. But cops in America aren't reasonable, because they aren't fuckin smart enough to hear what you say, when for example you say, "Yes, I left my driver's license in the house, which is right there, I'm just sitting in my car listening to music. Can I please go inside and get it for you?" No - way too much to process. Or when you say, "Is she under arrest, because if so I want to call her lawyer," and they crack you with a nightstick and chase you up Cahuenga Blvd while you're yelling at them for their badge number. Not that either of those things happened to me in the last ten years. But uh, fuck the police and fuck the government that chooses to arm a bunch of meatheads and send them out to lord it over citizens. I could tell you some stories about the cops my dad defended as a public defender, the ones from Culver City who only pulled women over to get blowjobs from them. But what would be the fucking point. You're all screwed. So much for a civilized conversation, now I've just let it all hang out on HN, and I'm sure I'll be attacked for it.

Sounds like you've been burned, sorry to hear that.

On the other hand, of course, if your house has been burglarized, you want just his guy on your side, ignoring the lame folks babbling inanities and looking for the one that won't meet his eyes, has a cut from broken glass on his hand, or too much cash in his pocket. Never mind socioeconomic theories. And when he finds him, he brings him in, even if it takes the truncheon.

Everybody thinks they know who should be running things. Scholars think scholars would do well. Merchants think they should rule. Soldiers want to rule all the time. Most of the time they're wrong.

It takes a certain mindset to survive any job. Assembly-line, sales, engineering. Even police work. You have to be pretty thick-skinned for that. More important, you have to go where angels fear to tread, confront argumentative people pretty much all the time, and be willing to be yelled at afterward because you gotta do what you gotta do, out on the street.

Give me a source for that which would allow for corroboration.

Because pretty much everything I can find on it has the whiff of urban legend, which folk like you repeat because it feed into their worldview.

"Some US police departments have set a maximum IQ score for new officers (for example: 125, in New London, CT), under the argument that those with overly-high IQs will become bored and exhibit high turnover in the job. This policy has been challenged as discriminatory, but upheld by at least one US District court. "

http://en.wikipedia.org/wiki/Intelligence_quotient

If there are factual inaccuracies there, please point them out. However just going by the "whiff" of something seems like you're opening yourself up to rejecting parts of reality that do not fit your worldview.

Edit: better source: http://www.aele.org/apa/jordan-newlondon.html and here is an interview of the man by CNN: http://www.postroad.com/news/2000/20000912-new-london-police...

I presume part of the reasoning is due to high IQ tests being too unreliable, and potentially full of people with borderline autistic personalities.

"No IQs above 125" is not the same as "No IQs above 90".

Ok, you brought it up. IQ tests test one thing - can you do an IQ test. Scholars love them, write them, and they filter for ... yes! Scholarship!

A human mind is multidimensional. The idea you can find one number to measure it is downright ridiculous. You wouldn't even buy pants based on just one number, say waist size. You'd get home, they'd be too long or too short. Obviously!

But folks buy into the IQ cargo cult big time. Totally baffles me.

> I think most of the verbal stuff in IQ tests wouldn't relate that much

Most software engineering jobs involve communicating with people. Customers, other engineers, other teams, leaders, etc. I would hazard to say that most effective senior engineers will spend at least as much time communicating with people as they will spend on individual technical tasks, if not more.

Communication skills are vital. If you cannot effectively persuade a team of people that your approach or point of view is correct, then your views, no matter their objective merit, will not impact the organization. Being an effective engineer requires having good ideas and also influencing other people to understand and adopt them.

A person can be an effective journeyman individual contributor engineer without strong communication skills, but to become a master with wide impact requires it.

My IQ is lower than it should be because I have what you might call "ADD". The final IQ score is an average of scores from a few different tests. I score high on some tests and average (which is actually well below average for college graduates) on others. The resulting score, then, hides my strengths and weaknesses.

Suffice it to say that I would have a hard time getting a job based on my IQ score because I can't do repetitive arithmetic operations as fast as other people.

Yes, my family has ADD in spades. Actually a help in programming jobs - the focus part is great. The intuitive jumping around can be just the thing when chasing a bug.

6 ADD kids from a farming community. One Entrepreneur :), an environmental researcher, a Nursing research specialist, two software directors (Case pumps, Intel) and one IT head.

Have you taken an IQ test? I have. It didn't strike me as a way of finding out much about a person, except perhaps whether they are able to do difficult arithmetic quickly. Also, people with lots of raw intelligence often find ways to misapply that intelligence in awesomely terrible ways. Smart in the little way, dumb in the big way.

I took a couple a few years ago, and the results roughly matched the test results from 30 years earlier. They weren't exactly the same tests, so the raw score numbers weren't the same, but the percentile number hadn't budged a bit one way or the other.

One of the tests had a lot of memorization - given a list of 9 numbers, repeat them backwards. Or 9 letters. Or a combination. Getting read "q9dh73b1wm6" and then having someone expect to hear you say back "6mw1b37hd9q" was quite nerve-wracking. Another test was "say as many names as you can in 30 seconds" (or something like that). Some of the tests appeared really odd, but the test-giver explained the thinking behind some of the questions later, and it was interesting.

You don't even need an explicit test. Just ask for their SAT and divide by 10. If post-1995, divide by 10 then multiply by 2/3. I still wouldn't hire purely on IQ though. An intelligent adult with no coding skill isn't that way for no reason. You cannot make someone into something they haven't much interest in being.

Your math for comparing SAT scores is off (what is the point of dividing everyone's score by ten?) There have been two main changes in scoring: in 1995 they recentered the scores which in effect increased many peoples' score on the order of 50 points or so (one can convert old scores here: http://2-bit.com/misc/satcalc.html). In 2005, they added a writing section so now your score is out of 2400 instead of 1600. Also, I think you'd want to compare peoples scores based on the current scale which would mean adjusting scores prior to 1995 so that they are recentered and either multiplying by 3/2 for scores before 2005 or discarding the writing portion for everyone.

But many people don't even take the SAT. At least when I was in high school (and likely still today), midwest universities tended to prefer the ACT. Thus, you're essentially looking for people who hit some percentile on national tests, similar to what one would do when applying for MENSA.

Also, as you and others have stated, this isn't really the best way to evaluate people for programming jobs.

The divide by 10 works as a gross estimated iq score (within 5 points) on the old sat.

It was about 20 points low for me.

Yeah it'd be pretty low for me too.

then again, there are several IQ tests, so that could contribute to the problem of having a generic conversion equation.

I can see the appeal in using SAT tests as they are pretty standardized but I don't think SAT scores would be a good indicator of whether or not you should hire someone for a couple reasons.

1. SAT scores can be greatly inflated simply by taking an SAT class. 2. The SATs are pretty bias towards native English speakers (as 2/3 of is reading and writing and the 1/3 that is "math" has a lot of reading comprehension required) 3. SAT scores aren't really a good indicator of the prospect employee's work habits or ability to play nice with others, both of which are pretty important points to consider when hiring someone.

Well. I suppose points 1 and 3 also apply to IQ tests.

What about people (like myself) who never took the SAT?

IQ is a terrible metric because "raw" intelligence is only one skill required to be a developer. It discounts creativity, people skills, personality, drive, domain knowledge, style, and learning speed.

If I got a candidate with a 140 IQ who was unpleasant, derisive, and had only coded in Pascal, I'd send them packing. It speaks to their drive, personality, and people skills.

A winning team is not built on intelligence alone.

Unfortunately, unless you also have a similar iq, the candidate will naturally seem unpleasant to you. Anyone would agree that they would not like to be around someone with an 80 iq. however for the same reason, namely differing values, attitudes, and thought processes those same people will not enjoy working with a 140 iq coworker.

Hey everyone, I just created a new service that sends you a new technical interview question every other day! Check it out at: www.InterTechTion.com

Reply on news.ycombinator.com