Hacker News Reader

Comment by jazzychad | Hacker News Reader

Hmm, I'm not sure I like the idea of intentionally trying to steer the candidate off on a tangent. It feels like a trap.

In an interview, the interviewer literally has all the power, and saying "no" or refusing to answer a question posed by the interviewer seems rather difficult.

This would be doubly hard for me since I like to teach (both my parents are teachers and I love to explain things to people so that they can understand them). If a person shows genuine interest in a topic by asking a question that might go in a tangent, I generally want their curiosity to be sated before moving on; in some cases a deeper understanding of a related topic will help educate them on the main topic.

Anyway, I think I would fall for this trap for both reasons (not wanting to refuse the interviewer and also wanting to genuinely teach someone). I'm conflicted.

However, I like the overall idea of the question. So much time during actual real-world development is spent communicating between coworkers about how a system works or will be implemented. I think being able to gauge a candidate's communication/knowledge transfer ability is very important and a good indicator of success.

View on HN

I'm glad you're conflicted! Because it is a trap, an intentional one meant to simulate the unintentional randomization that happens in real life at work. I'm sure we've all had that thing happen when someone threatens to accidentally scramble your work queue because they don't understand the problem domain as well as you do, so you need to be able to steer and guide the work requests that are coming your way and push back when it doesn't make sense.

My dad's a teacher too and boy does he love to lecture. About anything - digital signal processing, the price of gas, whatever - and that mode of communication is useful in its own context, but not so much when a team of people are trying to get something done together.

If it makes you feel better, I fell for the trap too. I asked Kevin for a demo of the question in action so I could see what it was like to be on the receiving end before I used it on candidates myself. I was the guy he referenced in the blog post that compared poker to backgammon and I was in the middle of explaining how Texas Hold Em taught me essential skills of entrepreneurship when he asked me how many board pieces were in a typical backgammon setup. I lost a good 20-30 seconds there trying to remember, until I laughed out loud and called him out on trying to randomize me :)

I think this would be ok if they said upfront that this is a simulation of a meeting and so to pretend that you are someone attending the meeting, and so explicitly take it out of the context of an interview. When I've been interviewed, we are often friendly toward each other and it's not uncommon for them to just want to chat and go on tangents, especially for throwaway questions that assess fit and culture. And you wouldn't know as the interviewee if this was such a throwaway question.

Basically, no matter what reasons you have for thinking people should do the right thing, you have to keep in mind that their reference set for YOU is all the people who have interviewed them in the past, many of whom were likely dumber and less competent than you. They are going to be acting accordingly.

Really? Because I've said no to folks five layers of management over my head, when I knew their comments to be technically inappropriate. I said it diplomatically, to be sure, but firmly and without hesitation.

I think the question is set up properly for the trap to be fair. Having been given clear responsibility for presenting a topic, a time limit, and the floor, I wouldn't think twice about directing the discussion.

Though I also don't perceive the balance of power in an interview to be that asymmetric, either. Usually they're courting me as much as I'm courting them. If anything, it offers a nice opportunity for a reverse test: management that has a problem with technically appropriate direction is not management I want to work for.

I have mixed feelings about this. I had an interview at Palantir with a self-described "front-end" guy, who asked me to write an algorithm to find a number between zero and infinity, where you could pick a number and you would get a response that would tell you whether or not the number was lower or higher than the secret number. The dude had a particular form of approaching the problem that was not at all the fastest form, in fact, he insisted on starting at zero and using an algorithm that was "2x," which he insisted was "fast enough."

I thought that a system which started at zero and went up by multiples of two was obscenely slow, and I hinted at this, but he insisted on having things his way. In the end I'm sure he wrote me a bad review and I wasn't offered the job (I didn't really want it anyways, given my impression that the team I was interviewing for wasn't that focused).

In any case, I didn't go right now and say this is dumb, but I hinted at a better way, and even this hinting lead to some conflict. In the end I didn't get the job. I'm conflicted if this was the best approach. I could have pretended to be dumber than I actually am and gotten the job offer at least, which at this present moment I could probably use for leverage in the search for contracts or a job, but I couldn't bring myself to accept his not particularly good algorithm (full disclosure: the process was also affected by the fact that my flight was delayed from the east coast, that I got 4 hours of sleep, the fact that I insisted in wearing a tweed coat to the interview, which may have impacted ever important perception of "fit," and that the interview in question was a "surprise" interview after several other interviews).

It's a binary search in infinite space. His solution expands the problem area until the number is in the space and then shrinks the problem area until it finds the number. A better solution is to pick an arbitrarily large number and only expand if you have to--otherwise you're shrinking from the very beginning.

That's how I would have answered the question, anyway. But yes, I've dealt with interviewers like this--typically it's because they don't understand the subject matter well enough to evaluate or accept an answer they didn't prepare for.

I disagree that that solution is unequivocally better. It depends on the distribution of the input data; if the expected value of the number is small, then you're better off starting at zero than dramatically overestimating.

Of course, either way the algorithm runs in O(log n) time.

Within the context of the interview question, the solution is specifically stated to be between 0 and infinity, and you can't surmise that the number is necessarily closer to 0. This is the error that my interviewer made. He was trained to deal with things within the context of situations closer to real life rather than the theoretical box he had established for the interviewing situation.

As for my case, I did attempt to start at an arbitrarily large number (but he wouldn't let me), and then went up exponentially (i.e. x = x * x), bi-secting the space once the upper-bound was found and solving from there.

My interviewer was sadly not that bright and didn't understand my solution, forcing me to re-write it -- incidentally, lecturing me on how wonderful O(log n) is along the way.

> the solution is specifically stated to be between 0 and infinity, and you can't surmise that the number is necessarily closer to 0.

Closer to 0 than what? Infinity? By any reasonable definition, every integer is closer to 0 than infinity.

I think the problem as stated is underspecified; I disagree that increasing "exponentially" (the solution your interviewer described would increase exponentially; your version is doubly exponential, I think) is always faster. Both versions have the same worst-case running time, and if you want to talk about expected running time you need to make assumptions about the data distribution. As a counterexample, if you start with an initial guess of 2 and keep squaring, but the true number is 2^1025, then you end up doing much more work than if you had doubled instead of squaring.

I can certainly agree with you that lecturing in an interview is unproductive. Not only is it understandably off-putting, but it doesn't give the interviewer any additional input to the hiring decision and therefore is a waste of everyone's time. (Unless one of the job requirements is coping well with being lectured to, I guess.)

> By any reasonable definition, every integer is closer to 0 than infinity.

Perhaps, but originally we weren't discussing 0, we are starting with an arbitrarily large number. This problem is exacerbated when you say:

> I disagree that increasing "exponentially"... is always faster ... Both versions have the same worst-case running time

Worst case, sure, but how do you compute average? You could take a selection of n number of integers from 0 to infinity and the number of guesses until you found the correct answer, but of course you can't do this because you can't easily get a random number to test with that is between 0 and infinity. That said, it is fairly clear that x * x is faster than 2x is faster than x+1, but you can only prove this to be true once you pick an arbitrary limit less than infinity -- and the benefits of the better algorithm are only evident as you approach infinity.

Seems like a catch 22. Perhaps there is some expert in set theory as applied within computer science that can provide the appropriate formal context for determining the better algorithm in this context.

As for O notation, it is an interesting tool but one that has no application within any realm of actual programming in which I have worked, and I have never bothered to learn it simply to dazzle people in algorithm-based interviews (perhaps because I do more nuts and bolts type work rather than optimization).

From the perspective of theoretical computer science, the interviewer was correct. I'll try to sketch out why.

------------------

1. In the initial probing for the upper bound, there is no point to grow faster than 2x each time

If we double the probe each time, we'll find the upper bound in O(log N) time. Then, we'll need O(log N) additional time to find the real answer. That makes the entire algorithm O(log N).

Suppose instead, in the initial probing we grow the bound faster, say by squaring each time. We'll find the upper bound faster, but we'll still need additional O(log N) time to find the real answer. So, we didn't really make the algorithm (asymptotically) faster - it is still O(log N).

(I glossed over some details in the explanation, but even if you work out the math exactly - which is not that hard to do - the conclusion holds.)

------------------

2. There is no point starting from a number greater than 1

There is no way to pick a "good" starting number - should it be 1,000? 1,000,000,000? 10^100? (10^100)^100? You might as well start from 1. That way, you guarantee that you'll find small numbers fast and large numbers still in an asymptotically optimal time.

------------------

As someone with fairly strong theoretical computer science backgound, I can see the intended meaning behind the interviewer's question and answer. But, it is a theoretical question. There definitely are a lot of highly valuable software developers out there who couldn't answer it.

Given that is a theoretical question that you are attempting to fit into an actual algorithm to ostensibly solve a real problem, doesn't the actual performance of the algorithm matter? I can see that within the context of theory of computer science there may not be a difference between the speed of the two algorithms, since both are "asymptotically optimal" (a term I'm not sure I entirely grok), yet within virtually any defined set of random numbers less than infinity the algorithm I provided will perform substantially faster.

I suppose I'm at a loss for words, since as a seasoned software developer without a deep background in theoretical computer science, it seems that I am being told to disregard my intuition based on experience in favor of a theoretical model that seems to ignore actual problem sets. Am I missing something?

I made a python script to time the different approaches:

https://gist.github.com/3839551

AFAICT, the parent and the interviewer is right because both methods run in roughly the same time because the time of the binary search is orders of magnitude greater than the upper bound finding part. I might have made one or more mistakes though.

If I have time later today I'll extend your snippet, but my inclination is that the differences in speed become apparent only with very large numbers. I still suspect my algorithm is about 40-50% faster if you start with a googol.

It seems that the problem involves two parts: 1) To find an upper bound 2) Then divide the remaining region in halves until the number is found.

The first observation that I have is that given that the secret number s is chosen, the first step can be completed arbitrarily quickly. One could use a function that rises arbitrarily fast. Imagine for example the function taking k to the Ackerman function A(2,2,k). That rises so fast it's incomprehensible, but really one could easily produce a function which rises faster still (the algorithm that picks s first!). The problem is, though, of how fast a fixed algorithm is for random s. If s is truly chosen at random from the positive integers, this leads to problems. Fix your putative algorithm. Suppose for the moment that it starts at 0 (it is not going to matter where it starts) What is the probability that the kth number that your function spits out is less than a random integer? 100% After all, how many integers are greater than any given integer?

Therefore, no growth function is any better on average at finding the upper bound than any other. Therefore step 1) is an insoluble problem. The problem should have been specified in some other way in order for it to make sense.

However, the second step involves log_2(n) time (in the worst-case-scenario and still O(log n) in general) where n is the upper bound --the output of step 1-- which means that the time to complete the algorithm is (Time of step 1 to find n) + O(log n). IF the problem made sense and it was the case that step 1) were soluble, then it would matter how fast step 1 is countered by the degree to which step 1 tends to overshoot the secret number --because overshooting by k has the penalty of log(k) extra operations.

Is there are framework in which question 1 makes sense? It would make sense if there were a given probability distribution on the integers (a function on the positive integers whose sum over all integers = 1, for example choose the function f(n) = 6/(n^2 * pi^2)) A probability distribution gives you a way of answering the question: "what proportion of the positive integers are greater than k?"

Excellent response. I think the framing element that was missing is if the random numbers were truly random.

This was my conclusion, that although the two approaches had the same average big O time, my intuition said it was somewhat faster to start from some number larger than 0.

In retrospect, I might have mentioned this and then gave the interviewer's preferred answer in code, just to keep things simpler.

My intuition says that they should almost always run at about the same time if you don't know anything about the distribution of guesses. Your algorithm has the advantage that it will find an upper bound for the number much faster than the interviewer's algorithm. HOWEVER, having found that upper bound, your algorithm will probably be much farther away from the actual number than the interviewer's, so you have to do that much more narrowing down. What it comes down to is being lucky that you're close to the number in your initial guess, otherwise, you're still talking logarithmic time to either find an upper bound or (in your case) lower bound for the number.

> Hmm, I'm not sure I like the idea of intentionally trying to steer the candidate off on a tangent. It feels like a trap.

And, yet, it's quite a realistic model of what happens in meetings and presentations all the time.

> In an interview, the interviewer literally has all the power, and saying "no" or refusing to answer a question posed by the interviewer seems rather difficult.

You could say the same with regards to managers or other higher-ups in real workplace environments. It's an important skill to be able to say no, or deflect gracefully.

In this situation, saying "I'm not going to answer that question" is the wrong response. Something like "that's an interesting question, but it will take more than 5 minutes to answer. I'd be happy to go into more detail later if you'd like." is.

I think it depends on the job. If I'm in project manager or manager mode, I might say "That's not relevant," but if I'm wearing my developer hat part of my job is providing information and so I'd be more than willing to jump down the rabbit hole.

> And, yet, it's quite a realistic model of what happens in meetings and presentations all the time.

Yes, absolutely. In a real work environment where we all know each other I would not hesitate to shut down a conversation that takes the discussion off course.

An interview is not like a real work environment at all. It can try to be a simulation, but it is still just a simulation where the candidate and the interviewer may not know each other very well. Maybe it's my southern upbringing (a topic of much mirth in my office), but I would consider pushing back on a stranger I am trying to impress to be somewhat rude. I have noticed that people out here in SV tend to be more aggressive than I am used to.

It's a realistic model, sure, but the stakes aren't the same.

In a meeting or presentation, you have a subject and an agenda and many people present who expect you to cover them.

In an interview, it's one-on-one. If the interviewer asks you A, then stops and asks you B, there's no reason not to move to B. If the interviewer is suddenly asking about B, they must want to know about it. There are no other stakeholders present who might care more about A. You didn't even know you were going to be talking about A until a few minutes before!

It's an important skill to be able to say "no" and stay focused on a subject. But this is a shitty test for finding out whether a person has that skill.

If you have to interact with customers at all, this is a very realistic situation. The candidate would get points for following up on the tangent question after demonstrating the ability to control the flow of the conversation.

That may not be key for all positions, but for more senior roles, and all roles which are predominantly customer facing, it's absolutely critical.

I disagree. There's no goal here.

Being able to say "no" is important, but so is knowing when to say "no" and when to say "yes".

If somebody tries to drag you onto an irrelevant tangent in a meeting, that's when you should say "no". If somebody is talking to you one-on-one and you discover that they actually care about something different than you thought, that's when you should say "yes".

The problem with this scenario is that it pretends to measure a person's ability to say "no" all while giving them no reason whatsoever to actually say "no".

If you want to test a person's ability to say "no", you have to put them in a situation where that's actually the correct move.

The OP doesn't give an example of these questions.

If you're explaining the game of 'Go' to me and I stop you and say 'Tell me about a time you achieved success' then I would expect 100% of candidates to stop talking about Go and start answering the new question.

On the other hand if I stopped you and asked 'Why do you think the counters are black and white?' I expect 100% of candidates to give me an answer that's fairly short and then continue with their original answer.

I can't think of a single question that would separate people into two groups.

There are definitely questions it that can discriminate this. Sometimes it will take a 2nd or 3rd question to be clear it's a distraction.

One example is a candidate that was talking about residential firefighting. I asked them about halon gas as a way to fight with quite a bit of curiosity, and they shut me down. I've learned to trust my insights, because I've seen them confirmed in other ways throughout the interview--it builds a pattern. When it doesn't fit the pattern, I am open minded that it's a fluke.

Ultimately, I think the "trick" nature of this is over emphasized by my blog. The single most telling factor is whether candidates can clearly explain something in essentialized form--that alone is difficult for most people.

I would never, under any circumstances, tell my interviewer to "stay on topic." How should I know what he believes to be relevant? If he no longer wants to quiz me on my abilities as a developer but would rather discuss the airspeed velocity of swallows, I would assume that he has a reason for doing so, and I would gladly share with him with my limited knowledge of avian aeronautics.

Also, the trap question could easily have the opposite "right response" as it does in the post and still be defensible as something important to learn about a candidate. Another interviewer performing the same test sequence might want to see if the candidate can change subjects quickly, roll with new circumstances, and not get flustered or stuck on a single, obsolete outcome.

"In an interview, the interviewer literally has all the power, and saying "no" or refusing to answer a question posed by the interviewer seems rather difficult."

If you feel like the interviewer has all the power in an interview, then it'll be difficult to negotiate your salary. You'll do things like give a concrete number when they ask you what your salary was at your last job.

The only time I would feel like an interviewer has all the power over me is if I was a new grad with no work experience.

An interview is a two-way street, you're trying to find out if you want to work for them as much as they're trying to find out if they want to hire you.

Doing an interview (specifically a tech interview in this situation) and negotiating salaries after you have an offer are two completely different things. I would fall for the trap, but I can (and have) negotiate a really great deal for myself in the offer phase.

The whole interview process is an implicit negotiation over salary.

They refer to the "leader" star candidates as the ones to say no, which makes sense.

Typically when hiring upper level managers you are interviewing someone who is going to be your peer - not someone who will be below you. Part of being a good leader includes pushing back on the people who are going to be your peers, if it means interfering with your tasks.

I have to agree. The question hinges on how the interviewee interprets the game being played in the interview. The interviewer seems to be offering the original instructions (explain something to me in five minutes) "out of character" and then gets into character and starts trying to undermine his original instructions. You're not really learning whether they can handle a meeting with interrupters, or whether they can figure out your silly games.

I loathe this kind of question in HR interviews as well. The far more common one around this way is for the HR staffer to just 'turn' on you halfway through the interview to 'see how you deal with conflict'. Thing is, that's really unnatural, and people do not do this in the outside world. I've worked in tech support, as a medical tech, with adults and kids and autistic kids, done phone surveys, done general customer support... never have I ever had someone 'turn' once I'd built up rapport with them. Sure, you could argue that they must be out there, but if so, they're such an edge case that it's pointless choosing on that basis.

I've also got to agree that 'teaching the uninterested' is hard for someone who likes to do the very thing he's asking.

Job interviews are nothing like the workplace for the interviewee. Things like silent time can be death - I've had interviewers ask things like 'take as much time as you need to think before answering' and then getting annoyed because I was thinking for all of 20 seconds...

"Leaders have the guts to say no." Are you hiring a leader or an employee? In my experience leaders tend to make poor employees and employees tend to make poor leaders. If you're looking for leaders don't expect many of them to walk through the door and sit through an interview. They're off doing something already.

When I first learned about this question, I thought this "trick" part of it was the most interesting. The more I've asked this, the less I care about distracting candidates. So many of them fail even to explain the basics clearly left to their own devices.

The interviewer does not have all the power. Any good interviewer knows this, and finding out whether the candidate knows this can be indicative of the candidates maturity, perspective, confidence, and leadership abilities. Not every position requires these, but some do.

"Don't worry about it. It's not important to the objective of getting you to understand X."

> It feels like a trap.

Reality is filled with traps.

> steer the candidate off on a tangent

Customers, coworkers, managers, do this intentionally or not, maliciously or not.

Any good candidate can do the technical job. Great candidates can survive and thrive in the work place.

> Customers, coworkers, managers, do this intentionally or not, maliciously or not.

None of these situations are the same as an interview. If you're providing an explanation at the behest of the interviewer, why not also depart on a tangent, also at the behest of the interviewer?

If you're explaining the value of your software to a prospect, why not also go off on a tangent at the behest of the prospect.

The role whose function an interview most resembles is that of a sales person. Having been in hundreds of sales meetings, I can say with total conviction that the ability to direct the conversation and deflect tangential questions is absolutely critical to being successful.

That said, every role doesn't require that skill, but OP specifically said that it's only for more senior/customer-facing positions.

Reply on news.ycombinator.com