Wacky interview questions
geekwire.com
geekwire.com
Here's what we do. Interested in feedback, please!
Via the phone:
* A few background questions (why are you looking for a job, what sounds interesting about what we are doing) (20-30 minutes)
* A bunch of topgrading questions http://www.smarttopgrading.com/ (maybe an hour)
In person:
* A few programming challenges (of increasing difficulty) that involve stuff the candiate is not expected to know (some algorithmic, most problem-solving type thing), and has to solve it via Google and Stack Overflow or whatnot, to make sure they can work outside of their comfort zone (I've had a lot of people freeze up at "Write a program to get Time from outside of the system", for example) (couple of hours til lunchtime)
* Then pair programming for the rest of the day to make sure we can all get along with this person (after lunch, and next day if possible)
If reasonable, we put a new hire on a 1 month contract before we make a final offer (we give a tentative salary offer at day one so they know approximately what it'll be but cannot judge equity grants until we have more time with them).
The point of questions like this is "Does this guy have any common sense?" Some of my least favorite hires were people who were whip-sharp at the details of their chosen programming language, but then when hired, would go off and do crazy and impractical stuff with it.
Whether or not the question accomplishes a good evaluation of someone's common sense is up for debate, but unless you're hiring drones, you do have to ask some questions that get a broader picture of the way the person thinks.
(BTW, your "pair programming for the rest of the day" is a great way of getting that. But pair programming doesn't/wouldn't work at any place I've ever been, for a variety of reasons.)
GROOOOOAAAAN
Maybe twenty years ago, and maybe if you were the individual to think up the question.
Honestly I'm so goddamned bored with these unoriginal blither-blather questions I consider them a strike against the interviewer and the company.
I'm curious--what do you ask in interviews to get at a candidate's common sense? It's not very easy to do.
Or do you think you don't need to assess common sense at all? Maybe it's not, but like I said, a lot of the hires I later regretted came down to something like that.
shrug If we're talking developers... The ability to drive to the location on time, answer questions without going blank are good indicators of common sense for me. Extracurricular activities, anything from volunteering at an animal shelter, to making civil war models to home brewing beer, are huge in my book because they are a good indicator of a grounded individual. That mixed with some pair-programming should give more than enough information. YMMV.
You don't need another developer to pair program to get that - essentially you set up a clean computer with their editor on it and your testing/interview environment, and they can hack away while you watch.
But yes, if there's some way to get a candidate to pair program on actual, real code with real developers, it's invaluable. Just make sure they sign a contract stating they are giving you the rights to the code, and that they are adequately compensated for their time at some reasonable hourly rate.
To your process, my thoughts:
- Open-book challenges like you describe (in which you can use stack overflow and whatnot) are awesome. However, why do you have them do stuff they're not expected to know? From what I've read, the only thing hiring studies seem to agree on is that the questions which best predict a candidate's long-term success are those that simulate what the candidate will actually be working on. Getting them to learn outside their comfort zone is good, but maybe you'd be better off doing that by finding problems you've actually had in the course of your work that are outside of their comfort zone.
- Short-term contracts are tricky. On one hand, I think they're the best way to test how well a candidate will work out. On the other hand, switching jobs is a lot of work and stress. If I had a choice between two pretty good companies, one of which was a 1-month contract and one of which was a normal job, I'd definitely take the latter. If I had to move to get your job, it'd make rejecting such a proposal even more of a no-brainer. If it works out for you, that's great, but I'd be really worried about losing excellent candidates.
- It seems like your remote screening is lighter than what I'm used to. At my current gig, we send candidates a challenge to be completed in their own time (3-9 hours of work)- we've found it's a great prelude to the phone screen with an engineer. It's pretty much a code sample for which we have an excellent reference (all the previous submissions). We've found it improves the quality of candidates that get to our on-site.
What I mean is that, I put together a few challenges with APIs we work with (or gems we've written, that sort of thing) and have them solve a problem that will require them to use their innate ability, drive, and Google/StackOverflow/etc. to solve the problem.
It's not something that they would be able to do without the internet. Does that make sense?
The contract thing is tough, although at least in NYC the market is so tight that there's really no risk of unemployment, but yeah, we wouldn't do relo + contract. We'd start them off as remote and see if that worked. If so, then relo; it's risky risky for both parties (especially in NYC) that it's kinda crazy to expect either party to agree to it.
The candidate has no idea what NYC is like, and the organization may be reluctant to fire a bad hire who just isn't fitting in because they feel bad. At least with a remote contract, no one has to move and we can see if we both like each other enough to get married. Then would come the offer + relo., or if it's not a good fit, end the contract (with 2 weeks notice at least).
Finally, I'd say the initial screen is a bit light. We used to have about a 1-2 hr. technical Q&A followed by Fizzbuzz-style questions, but that had a low false-positive rate so we got rid of it.
That is to say, the background work I did (googling them, stack overflow, github, twitter, their resume, etc.) was a pretty good filter at keeping out the obviously underqualified.
However, I'm considering putting it back as it helped give me a more nuanced picture of the candidates that I end up doing as part of the contract, and it might help to have some first-impressions ahead of time (but I am always worried about false first impressions clouding my judgement).
I'd have to go with "Take this job and shove it". There's nothing worse than places that mandate you be "fun" and "offbeat". If I am fun/offbeat, all you as an interviewer have to do is make me comfortable enough to show it. If you can't do that, why do I want to work for you?
If http://www.glassdoor.com/Interview/LivingSocial-Interview-RV... are to bee believed this company are wasting a lot of peoples time.
That said, there are some doozies in this list. For example: "On a scale of one to ten, rate me as an interviewer".
Um, okay but only if the next question is to rate you as an asshole.
I guess if the person being interviewed is 30+ and working in software development and hasn't been exposed to Fermi questions then maybe that's a red flag, but other than that I've seen no indication that answering these types of questions well correlates to real world performance.
1. Mass of the earth in kg 2. US GDP 3. Number of carriages on the London Underground
I then had to give a 95% confidence interval for each one (i.e. an interval that I was 95% sure the true answer lay in) and then asked, for each estimate, which bet I wanted to take:
1. Bet that the answer is inside the interval at odds of 10 to 1 on. 2. Bet that the answer is outside the interval at odds of 30 to 1.
There's a lot of work that has to be done there, and potential for a very interesting discussion (this was for a trading/research role, rather than a software developer role).
All of those involve memorized numbers, and most require physics knowledge. But I have a feeling that the more complicated ones would impress them more, and make them more likely to hire me. If that's the case, then my strategy is not to give the answer which I know, but to exaggerate the difficulty by hiding my actual knowledge.
GDP? It's about the size of the debt, which is something like 7 trillion, so I would have estimated $10 trillion. (Actually, the debt is $12, and the GDP is $15.) I might have double-checked my answer because I think the budget is about $4 trillion, and 20% of GDP giving $20 trillion. (Actually 3.54 trillion and 22.6% GDP).
How is this memorized knowledge useful? Especially since I don't know how the GDP is actually calculated? Nor the difference between GNP and GDP. So this mostly tests to see if I've paid attention to the economic news. It's almost like asking me if I know the hottest temperature ever recorded in Death Valley.
And as for the London Underground - too many variables for me to even estimate. Turns out to be 4134, and that what I thought was part of the Underground was actually rail services. If forced, I think would have said: 20 rail lines? (Actually, 11.) At rush hour, 10 cars separated by 5 minutes distance, for a total of about 90 minutes per line? Which gives 3600 cars. I probably would have rounded up to 4,000 for spares, and then been amazed at how close I was. (Ahh, I forgot the factor of two because trains are going in both directions. And I think the inter-train spacing is less than 5 minutes on some of the lines.)
But this only works because I've been to London and made those observations. Without that experience, I would be off by an order of magnitude.
So there you go - this proves that I have a degree in physics, that I follow the news, and that I've visited London. It helps with the job you applied to because ... ?
(And if you say it's because I can reason about these sorts of problems, then, see "have a degree in physics" and add "and also math.")
If the candidate performs well on this question, then they have shown:
1. They have the ability to reason about problems in a potentially new domain.
2. They have the ability to pare down a problem to what is important and what can be neglected.
3. They can make good order-of-magnitude estimates (very important for an equities trader).
4. They are aware of the typical size of their errors, or can estimate them.
5. The are aware of the typical sources of error.
6. They understand the notion of a confidence interval and can apply it to real-world situations.
7. They can reason about bets with different characteristics (e.g. different expected value, different skew characteristics) and choose a good portfolio of bets.
I think that knowing how GDP is calculated is useful information for someone working in finance. Especially someone who is going to have to trade through economic announcements. Would you hire a trader who didn't follow economic news?
PS re "shows you've visited London" - the interview was in London. I assume if it had been in New York, they would have asked about the New York metro.
There is nothing, for example, which probes my experience with statistics.
Here's a more in-depth test. We run an medical experiment on lab rats. The protocol says that we will experiment on 20 rats, and that a mortality rate above 30% is too high so we must stop the experiment. After 5 rats, 2 have died. Should we stop the experiment now? Why or why not?
One could look at that and say that a 2/5=40%>30% so of course it should be stopped - it's higher than the allowed rate.
A better response is to point out that if 10% is acceptable, then there's a 0.9^3 * 0.1^2 * C(5,3) = 7% chance that this is an expected statistical fluctuation, and that the experimental protocols should have given some guideline on how certain one needs to be in knowing if the mortality rate was too high.
An even better response would be to point out that this means the first three rats lived, and it was only rats 4 and 5 which died, as otherwise the experiment would have stopped earlier under this protocol. In that case the possible experimental states are: 1) first rat died, so the experiment stopped = p. 2) first lived, second died, stop = p. 3) first lived, second lived, third died, stop = p. 4) first, second, third lived, fourth died, fifth lived = (1-p). and 5) first, second, third lived, fourth and fifth died = p giving (assuming I recall how to do this correctly) p+p+p+(1-p)+p = 1.0 or an estimated mortality rate of p=0.3.
In other words, this experiment is allowed to keep on going under the given protocol.
But this sort of modeling awareness is not present in the three tests you mentioned, despite being directly relevant, I assume, to what a trader does.
I watched a once-coworker give a 30 minute lecture to high school students on how to do these giant puzzle/estimation types. His trick was to take the geometric mean of reasonable min/max bounds at different stages of the estimation.
Example question: How many flights occur in America in a day?
You can start from the bottom, with an estimate of the number of people who fly, then the number that can fit in a plane, et cetera. Alternatively, you can start from the top at the number of airports and their flight capacity. With a bunch of sqrt(minbound*maxbound) and reasonable guessing, the students (via groupthink) ended up with two roughly matching estimates that were within an order of magnitude.
My response would be, "If a penguin in a sombrero walks through the door and starts talking, the most plausible reason for why he's here is that you've spiked my coffee with LSD."
He says "deslizarse". He's there because he is my power animal.
My guess is that they want to learn your sense of humor, to know whether you'll fit with the team.
The correct response to this question is to say "Could I please hold your pen?", and wink slyly.
I consider it stealing as much as I consider using the toilet paper they provide for me stealing.
Go see an ENT specialist.
I know the correct answer is supposed to be 0, but the hour hand moves closer to hour+1 as minutes approach 60. I guess I would ask for a compass.
Another fave is: “How many quarters would you need to reach the height of the Empire State building?”
My answer would be: "What's the price of admission?"
30 degrees between hours. 10 is at 300 degrees, 11 is 330 degrees. 50 minutes into the hour is 5/6 of 30 or 25. 330+25 == 355.
Certainly when I had my Google on-site for an engineering role I was spared the infamous questions that crop up in articles like this from time to time.
I've also seen various variations of the Fermi questions before, though maybe not exactly these ones. Like half of the questions listed there fall into that model: Estimate how many XYZ are there in ABC?
Of the others, “How would you direct someone else on how to cook an omelet?” seems pretty reasonable in terms of sussing out communication skills, but a lot of the rest are just insanely stupid questions, IMO.
I like #12, just because it begs to be answered with "perfectly".
"One too many, if you make me work on J2EE."
Sorry what was that again? I forgot what you asked
Amazon: Depends on how much PHP I have to write.
Google: Define cow. Does it only apply to the common animal or does it also apply to the slang used to refer to overweight people?
Zillow: Define success.
Expedia: Which language version for the T9 keypad? In what language do the words have to be found?
“If you were to get rid of one state in the US, which would it be and why?
Define "rid".
“How many cows are in Canada?”
Define cow. Does it only apply to the common animal or does it also apply to the slang used to refer to overweight people?
“How many quarters would you need to reach the height of the Empire State building?”
Define the specific quarter used for the exercise. Also, tell me the temperature at which the exercise will be done.
“A penguin walks through that door right now wearing a sombrero. What does he say and why is he here?
What type of penguin, and sombrero?
“What songs best describes your work ethic?”
What genres are allowed?
“Jeff Bezos walks into your office and says you can have a million dollars to launch your best entrepreneurial idea. What is it?”
Define best.
“What do you think about when you are alone in your car?”
Is the car ON? Am I driving somewhere? If so, Where?
“How would you rate your memory?”
Which type of memory (short/mid/long-term?)?
“Name 3 previous Nobel Prize Winners.”
Can it be for the same Nobel Prize?
“Can you say: ’Peter Pepper Picked a Pickled Pepper’ and cross-sell a washing machine at the same time?”
When do you want the Washing Machine delivered?
“If we came to your house for dinner, what would you prepare for us?”
Are you allergic to any type of food?
“How would people communicate in a perfect world?”
Define perfect world.
“How do you make a tuna sandwich?”
Which type of tuna sandwhich?
“My wife and I are going on vacation, where would you recommend?”
Are you going together?
“You are a head chef at a restaurant and your team has been selected to be on Iron Chef. How do you prepare your team for the competition and how do you leverage the competition for your restaurant?
When is the competition?
“Estimate how many windows are in New York.”
Are we refering to regular windows or computing devices running the operating system Windows? Plus, define Window. Is it an opening in a wall? Does it need to have a glass plane?
“What’s your favorite song? Perform it for us now.” Define favorite (as in top 10 ranking, overall, etc.). Define perform (sing, sing+dance, dance, mime, etc.).
“Calculate the angle of two clock pointers when time is 11:50.”
AM or PM?
“Have you ever stolen a pen from work?”
Define stolen.
“Pick two celebrities to be your parents.”
Define celebrities.
“What kitchen utensil would you be?”
Define utensil (mechanical, electric, electronic, etc). Define kitchen (commercial, American, European, etc.).
“If you had turned you cell phone to silent, and it rang really loudly despite it being on silent, what would you tell me?”
Which cellphone (brand, type, OS, etc)?
“On a scale from one to ten, rate me as an interviewer.”
Does the scale go from best to worst or otherwise?
“If you could be anyone else, who would it be?”
Define be.
“How would you direct someone else on how to cook an omelet?”
Are they disabled in any way?
It doesn't matter.
Using when implies that I do the calculation at the exact point in space-time, and not calculate the angle of the clock pointers at the moment the clock marks 11:50.
:)
I know you're joking, and there is value to getting the interviewer to clarify the question and confirm assumptions.
But some of your responses ("define be") are nitpicky to the point of "this guy wouldn't be any fun to work with".
But I always show up for the weekly lets sing happy birthday to those who had their birthday this week. And sometimes I'm the one who brings the cake. That's the definition of fun[1].
[1] In corporate America.
It definitely doesn't mean "fun only during non-work bullshit".
>Are we refering to regular windows or computing devices running the operating system Windows? Plus, define Window. Is it an opening in a wall? Does it need to have a glass plane?
>“How would you direct someone else on how to cook an omelet?”
>Are they disabled in any way?
>“If we came to your house for dinner, what would you prepare for us?”
>Are you allergic to any type of food?
>“My wife and I are going on vacation, where would you recommend?”
>Are you going together?
I actually quite liked these answers, and I would think positively of a candidate who asked them (assuming I would ask the questions in the first place, which is unlikely).
While "Define this that and the other ad nauseum" is probably overly pedantic, the above do illustrate an ability and propensity both for analysis as well as even perhaps empathy - both, IMO, meaningful skills when developing a product.