Hacker News Reader

Comment by arafa | Hacker News Reader

Brain teaser questions don't perform well in studies and should be avoided: http://qz.com/378228/google-is-over-those-ridiculous-brainte...

Google suggests doing behavioral interviews instead (which I've also been trained to do and have gotten good results). Technical interviews and such may be necessary as well, of course.

View on HN

When I was interviewing, I loved brain teaser/market sizing questions because they were completely formulaic and easy to ace. They test nothing but "does this person know how to answer this type of question": take your time, think out loud, come up with some type of structure, try not to screw up any basic math.

Now that I'm an interviewer, I stay away from them completely. :)

I would say I would -never- base a hiring decision on someone who was poor at answering a market sizing or logic question. However, I do like to see candidates with determination to try and answer, explore multiple possible answers, and remain cool during their reasoning. To me, it’s reflective of your working style and PM’s must remain calm, rational, constantly thinking of all possible options to solve a working challenge.

But is there data to support that these questions are effective in finding the types of people that you want? Without concrete and consistent answers to questions, you'll be exposing yourself to bias (subconsciously encouraging people like yourself or seeing their vague answers in a more positive light). You'll also make more of the interviewing experience dependent on what you happen to think of and the mood you're in at the time, increasing the inconsistency of your interviews.

I know the intuitive idea is that you'll get to see how the candidate reacts, but that can be done in a less arbitrary way (and in a job-dependent context) through behavioral interviews and the like. Brain teasers are also often poor candidate experience, since they can feel unfair and unrelated to the work. If estimation is a key feature for PMs, it should be part of the interview process with well-defined procedures.

How does one train in behavioral interviews? Is there a methodology around it that can be adopted by someone who wants to build it into his startup's hiring process?

You could adapt something like this to fit your needs: http://www.businessinsider.com/google-laszlo-bock-interview-...

This will give you the general idea of how they work. I'm sure there's classes and other resources, but we already have a training program at work so I haven't delved into them as much.

I do a lot of PM phone screens. I don't ask brain teasers but I do ask basic math problems (i.e. drake's equation type problems). They are very good for screening out candidates who won't be able to do the analytical part of the job.

1) That's not a "basic math problem", it's exactly the kind of brain teaser/gross estimation problem this thread is saying doesn't work. 2) You're going to need to explain to me how estimating the number of communicative extraterrestrial civilizations in the galaxy (or a similar question) is an indicator of how well a person can understand a product's features, tech stack, and customer needs. Unless your product is something related to statistics, you're just screening out people who aren't like you.

One variant I ask is 'estimate the number of delivery drivers needed to start a same day delivery service in a city'. Do you think this is a brain teaser or not? This is clearly not a statistics product...

I find PMs need the ability to break down problems into manageable pieces to calculate, and estimate things with lots of ambiguity. How else do figure out whether an idea is viable or not, and worth spending more time on going deeper on?

And I meant to say fermi problems, not drake's equation.

Yes, that's an interview brain teaser of the exact variety Google and others have found doesn't indicate much in the way of actual job capability/fit. As someone else in the thread pointed out, anyone can look up the methodology for answering these types of questions in about a minute, and be able to tackle all of the variants easily with little actual thinking.

I agree that product managers need the ability to break down big problems (or requirements/client requests/company direction) into manageable pieces for their team to work on. However, there are better ways to gauge this ability, such as...

- Ask directly for an example of how they have taken a big problem or assignment and broken it down in the past

- Give them a real-world big problem you've had recently and ask them how they would break it up into team tasks/user stories/whatever you use

- Ask them to explain their methodology for determining how long something will take to build or test (ex. some people love planning poker, other people love logarithmic buckets, other people base estimates on experience, and others do something completely different)

Basically you'll get a lot more value out of interviews if you ask directly for experience or process they'll be using rather than trying to back into a skill estimate using a brain game.

> They are very good for screening out candidates who won't be able to do the analytical part of the job.

It's very unlikely that you have any way of knowing that, since you don't know how good people who fail would have been at doing the analytical part of the job.

Reply on news.ycombinator.com