If companies interviewed translators the way they interview coders | Hacker News Reader

If companies interviewed translators the way they interview coders

medium.freecodecamp.com

366 pointskissgyorgy254 comments

I was once asked a very very specific question about a conflicting setting in an nginx config and the dude actually wrote the config on the whiteboard and asked me how I would make it work.

I stared at it for what felt like minutes and then said, if I looked in your search history would I see you looking this up on stackoverflow?

The guy said "yes" and I said I would make it work by asking you to send me the link to the stackoverflow answer.

He laughed and said "you got me".

Same company different interviewer asked me to explain the "pros and cons of Java vs Rails."

I turned the job down.

I dunno, I think Java vs Rails is a great question if asked right. There are large meaningful and practical differences between the two and every non-jr programmer should at least be able to talk about the technical and business merits of using one vs the other.

There isn't a right answer, you basically just start shallow and probe until either you or the candidate can't go any deeper. This question could lead you into exploration around the JVM, concurrency, memory management, typed vs un-typed, choosing libraries, toolchains, dependency management, strategies for persisting state(i.e ORMs), back-of-the-envelope performance, etc

I'm pretty sure I could get a full 45 minutes out of that question.

So you mean Java vs. Ruby? ('Rails' being 'Ruby on Rails')

Java vs. Rails is a bad question as it's comparing two things that aren't equivalent. One is a language, the other a web-application framework.

It's not a "bad" question, but your reply is a fine starting point for a good answer:

"Well, for starters, Rails is a framework while Java is a language. Granted, Rails is THE framework on Ruby, so I can tell you about the differences between Java-based webdev, vs Ruby/Rails. With R/R, you have convention over configuration, which means blahblahblah and is good for getting started but at times you run into blahblahblah. And community blahblahblah, etc. Java, OTOH, has stating typing, which gives you guarantees blahblahblah but there's no standard web framework like on Rails so you'll spend a lot of time researching not-so-great choices, etcetc...."

Sure, there's a good amount to discuss, but I would argue it's too open ended.

I could also, as the grandparent suggests, talk about that particular topic for a long while. But as an interviewee it would suggest to me that they possibly are just spewing out words they heard around the watercooler. Either that, or it's a 'trick' question, due to the fact that they are not comparable.

I expect chefs could talk all day on the question: "What are the pros and cons of flour vs. forks?" That doesn't make it a good interview question.

I believe there is no such thing as too open-ended a question to ask an interviewee. If you're asked something open-ended you have a glorious opportunity to define the scope of that question yourself, and to frame it in terms of things you know. Better still: you get to throw the ball back in their court by asking supplementary questions.

The point of such an interview question is to foster dialogue.

Yes. I agree with this. A strategy I employ when interviewing is to run with the first open question. It informs the interviewer about my knowledge and experience and gets me out of the majority of senseless tech questions.

Better analogy would be "What technology would you choose for building a bridge". You should compare software engineer to construction engineer or architect and not to a cook.

Valid quibble.

> every non-jr programmer should at least be able to talk about the technical and business merits of using one vs the other.

So there is no place in this world for a senior programmer who isn't particularly familiar with those two languages?

Why are they so special?

I would expect a senior engineer working on Web technologies to be at least familiar enough with what both Java and Rails are to be able to start that conversation.

It doesn’t require an in-depth knowledge of either, but this question would serve as a great way to get a basic level of an applicants knowledge.

Meanwhile I could easily imagine a senior web developer with 15 years of programming experience having used JS, Python, Perl, PHP, ASP.NET, and a dozen non-web languages but never a single line of Ruby or Java.

I have never used either Perl Or ASP.NET. I would be comfortable talking about the difference between them at a high level regardless. I don’t think it’s unreasonable to expect a senior developer to be familiar with the tools that are used in the trade generally, even if they have not used those tools themselves.

I think it is quite unreasonable.

I honestly think that talking about some tools someone never needed is a terrible way to get the feel for a basic level of that applicant's knowledge. And it really doesn't tell you anything about them as a programmer. I think even fizzbuzz and linked list reversal would be more telling, and that's not much.

Going by the translator analogy, I feel like it's asking say an Asian translator (who might command a few eastern languages with decades of solid experience) who never studied English and French to talk about English and French. Because in your world everyone skilled translator knows enough about English and French to at least have a useful conversation about them. It smells very elitistic. And pointless.

We'll just have to disagree on this one, I'm afraid.

I would consider it a necessary job requirement for a senior developer to know about common technologies in the field they are working in. If that meant working on web applications, I would expect them to know what both Rails and Java are. I don't need them to be able to program in them; I need them to know what they are, why they exist, some of the key facts about them, where they might most commonly be used, that sort of thing.

Over-specialisation to the extent that developers are unaware of technologies outside of their direct knowledge is a warning sign about a developer's skillset, to me. How can I trust that a developer is picking the correct tool for a job if they aren't aware of what tools are available?

It's possible to be aware that certain tools exist and have an idea about what they're used for, and yet know little enough about them to butt into a discussion and give any sort of fruitful, educated opinion about them.

My imaginary senior web developer knows more than a dozen languages. It is almost a given that one of these languages is technically sound and a reasonable choice for a given task, and that developer should be able to pick that language just fine. However, if there are other compelling reasons to look into other languages (for example because company wants cheap coders and think there's a large pool of cheap talent skilled in language X, or because the company needs to leverage the ecosystem surrounding language Y), that same senior developer is more than capable of doing a bit of research and evaluating these languages.

You almost sound like one of these interviewers who think that a programmer who can't implement a red-black tree off the top of his head and state its worst case insertion complexity couldn't possibly be capable of looking it up and evaluating the relative merits of different data structures -- and thus cannot be trusted to any position where he might have to pick an algorithm. Nevermind if he could implement two dozen other data structures in his sleep.

Over-specialization? Seriously? A programmer who's used more than a dozen programming languages is over-specialized because he can't discuss the two you want to talk about? The stench of elitism is only getting stronger here. You're assuming the worst of a programmer because his world doesn't revolve around the same things yours does, and you are assuming that despite their experience they probably will not try to or cannot educate themselves further.

Unfortunately I see this sort of prejudiced attitude everywhere... "I think this is important so if you don't know this, you can't possibly be any good." Such statements reveal a rather narrow world view.

Less of the personal attacks, please. I think I'm discussing this in good faith.

Yes, that's probably true. If that were the case, I would expect an answer like "I don't know anything those beyond the fact that it's a cross-platform language that runs on a virtual machine and Rails is a framework for Ruby. But I do know a lot of Python and Go, which have some similar characteristics. We could talk about the general difference between compiled and interpreted languages?"

These questions serve as entry points. I'm not expecting a developer who doesn't use these tools to have an in-depth knowledge of how everything works, but if the question kills the interview to the point where someone is unable to move forward then that is a warning sign.

You almost sound like one of these interviewers who think that a programmer who can't implement a red-black tree off the top of his head...

No. I absolutely would not expect that, and those questions are stupid. I would expect a senior developer to know what a tree is, and maybe what a red-black tree is. I would expect them to know what operations can commonly be done on a tree, or what use-cases they might have. I would expect them to be able to, say, contrast a tree with a hash table. If a developer said, however, "I have never used a tree or a hash table and have no idea what you are talking about" then I would also consider that to be a warning sign. Do you see the similarity?

You are misrepresenting my view; see above for more details. I do not think it is elitist to expect people in senior roles to know about common tools used in that role. I would not expect them to be specialists, but I would expect them to understand the state of the field and be able to discuss that. I mean, we're not talking about obscure technologies here!

That is an unfortunate misrepresentation of my view, so I'm sorry if I've failed to communicate it effectively. I guess from what you've said above that you think I'd place much more weight on the technical content of that answer than I would in practice.

Perhaps I have misinterpreted you or read too much into what you said. I'm sorry if my response came across as a personal attack.

All these threads about interviewing over the years have definitely left me with a bitter taste, mainly due to all the people who think they're clever and have figured out the most important thing, that one thing by which all applicants' worth can be tested.. if they just know about this one little thing. And in doing so, they effectively trivialize the software industry as a whole, failing to realize just how wide and deep it is, completely oblivious to how different developers with up to two decades of experience even in the same subfield may have walked rather different yet fruitful paths.

Again, I'm sorry.

I agree, question about choosing between Java and Ruby is a good question for senior software developer position. Choosing between technologies, platforms, frameworks is important and responsible part of the job.

Valid point. And we did have that discussion. They were a rails shop and not considering Java for anything ... so it was very much a very random question in regards to the position.

He also wanted the pros and cons of them verses each other based on his wording.

My original answer was "the obvious pro is a death match is always interesting"

One of my friends told me that when he was interviewed at his current company, they asked him a similar question, and he said he gave a long complicated answer that basically amounted to "I would google it."

He got the job, so there you go I guess.

Gotcha style questions like that pretty much just ask "have you been in the exact same scenario as me last Tuesday?"

My interviewing has boiled down to 30 minutes of shooting the shit to see what the person is like and to make them more comfortable and then a pair session with a ticket I haven't had experience with so we are both on approximately the same level.

My goal in the session is to see how they interact with an unfamiliar project and what kind of questions they ask and their discovery process of learning our business domain.

Your interview process is unfair.

1. The shoot the shit session is guaranteed to be driven by your personal biases.

2. Working on a different ticket with every candidate ensures you can't possibly compare one candidate's performance to another's.

Totes, but what interview is fair?

I also send the same screening test depending on role so I do have a baseline, although I'm much more interested in our interaction during the pair session.

I'd rather have an informal discussion that demonstrates competence than being quizzed on meaningless Googleable stuff[0].

[0]: https://news.ycombinator.com/item?id=14570309

I said the same thing to an interviewer once when he asked me about the syntax of some Entity Framework command and he looked at me like I had cheese coming out of my mouth.

I didn't get the job, but I'm glad. His reaction was a red flag as far as I'm concerned.

edit: I didn't even notice the "rails" until after I wrote this. My brain filled in "ruby". I agree that java vs rails is stupid.

I love the compare/contrast style of questions between languages. I try to not phrase it as "pros and cons" however. What I'm looking to see is the breadth of their knowledge and what insights they've picked up from hard earned experience. Also, showing the ability to think critically about different tools demonstrates that they understand when a hammer is appropriate vs a screwdriver.

I don't care what they think is better or worse, I want them to show me that they've traveled the world, so to speak.

It's not completely stupid. My answer would be:

If you need something up and running fast, but don't care about performance per instance, such as a marketing landing page use RoR.

If you need something very performant per instance and are willing to pay for it, such as a enterprise application or large web site, use Java.

If that's not an acceptable answer, the interviewer doesn't know what he's doing.

The "stupid" part is comparing Java to Rails, not Java to Ruby. IOW you're comparing a language to a web framework.

I agree with everything you said.

Generally, I never interview about things "I know". I want to know how the candidate approached his/her project - the choices s/he to consider while designing the solution. It gives me far greater insight into the candidate's mind than merely asking "whiteboard questions".

I don't understand how Stack Overflow is so popular for these types of questions. Usually the documentation is a much better source. So yes, using Stack Overflow instead of the documentation is a red flag for me.

I think it's because Google somehow manages to index stack overflow better than the documentation. That, and stack overflow only answers real-world questions while the documentation tends to give equal weight to rarely and commonly used features.

The reason is the text of the questions, which is what people type into the Google search bar. Documentation never has questions worded in the way people would commonly ask it.

For this reason, I wish they would allow redundant questions to be asked, if the phrasing is even slightly different. But the rule-mongering mods always remove it if they can.

I didn't think dupes on SO were removed, rather they are marked duplicate, linked to the original and closed, precisely because of discovery.

Perhaps SO could include the titles of questions marked as duplicate somewhere in the markup so that Google indexes it?

Which is exactly what they do:

Should duplicates be deleted?

In general, no: most duplicates stay around. Having multiple copies of the same question with different wording is useful as search fodder, because people looking for an answer may use different wording too.

https://meta.stackexchange.com/questions/10841/how-should-du...

That's good to see that they actually have the right rules, but I've used it many times where my questions were closed simply for appearing similar to others (when I did not even find the others in my search results). I suppose the mods can be overzealous.

Being closed is not the same as being deleted.

Exactly.

It's not due to some Google-fu though. People ask questions (on SO) in the terms they think about problems. But documentation is written according to the structure of the thing being documented.

So SO ends up being a more direct mapping from problem to solution. Google simply captures what's there (in both cases). It's certainly good to consult the authoritative docs in many cases.

Also, SO optimize for Google (it's a large part of their traffic and they are a successful business) while most documentation efforts don't have the resources or inclination to work hard on SEO.

That always annoys me. Instead of getting a link to the official documentation, I get 5 links to vaguely related StackOverflow questions, the most useful answer of which contains the link to the documentation.

the fact that you don't summon the official documentation by google search is exactly why the indexed questions are beneficial to you: being a glorified pattern matcher, google would spit out the docs only if provided by the documenatation precise wording itself, which if you knew, you wouldn't needed to find

besides, google is a knowledge base index, if you want to search a documentarion website you can both both inform google via site: or use the docs own search system which most have as of today

Google's got a lot of smart developers who you'd think would have figured out how to surface original documentation if they really wanted to.

why tho? it's easy enough to add +javadoc (or equivalent) to a class name, no need to second guess the user input

It depends, documentation is organised very differently than SO. If you want to read up on how X works in general, then documentation style is useful; but if you want to find out a single narrow application of X, then stackoverflow style is useful.

For simplified example, if you want to find out what flag -wtf does in some unix utility, the man page would be useful; but if you'd want to find out which flags should be used to achieve WTF then you'd often need to read the whole documentation to find out - stackoverflow serves as a "reverse index" in this case.

The documentation is great when you know what you're looking for.

When you don't exactly, SO is a good pointer to the part of the documentation you didn't know.

yes back in the day IBM and DEC had good documentation today not so much

Not knowing why SO is so popular is a red flag for me.

Good SO answers usually link to the documentation. Often quicker to get to it via SO than to trawl through the docs hoping to find it.

It's not on SO people search for their problem; it's Google. If SO comes up before the official documentation, then I guess by at least a quantifiable metric, SO is not that much worse than the docs, after all...

If the doc was so good there wouldn't be a stackoverflow to begin with. The documentation is a better source, granted, but most of the man pages I read over the years seem to have been written by people who don't know how to communicate and explain concepts. It's like giving someone a mendeleev's table and expect them to be able to set up a chem lab and conduct experiments.

Well for me, I use stack overflow to translate. I want to do this in this language / framework, how do I do it?

I'm sitting at a bar in Turkey. I want to know how to say, "I want a beer," in Turkish. I know how to order a beer, drink the beer, and pay for the beer; I just want to know how to do it in my current circumstance.

I think it'd be great if there was a google JS snippet documentation sites could include to have a list of dynamic gotchas or faqs.

How many times have you read docs and thought "well that's relatively straightforward" just to end up googling an error message 10 minutes later and end up on SO.

I think a real solid skill is being able to ask google the right question when troubleshooting something.

NOT using SO it's are flash. Use all your tools. But SO answers are generally easier to understand than the documentation, or the docs don't actually explain the issue you are looking for, or they are out of date.

For better or for worse, I've now become abrasive when asked interview questions in such a scripted, artificial manner. Because I no longer want the job at that point, I can respond more boldly, e.g.:

"Where do you see yourself in 5 years?" "I don't know. Are you reading these questions from a textbook? FYI they're not very effective if you want to find someone who will do the job."

"What's your greatest weakness?" "Trick questions in job interviews."

This is obviously not good advice; I have just reached a point in my life where I will not be made to dance to the whims of the interviewer, despite which the job would likely go to one of the employee's friends rather than it being given to me anyway.

One of the few merits of this approach is it tests if you can be frank with them or not. If they get offended at your lack of sucking up, then you probably wouldn't want that job anyway, because I find that if you suck up during the interview, you will find yourself struggling to maintain that ideal forever if you do get the job. It's better to be upfront about what things you actually care about.

Don't judge a company by the interviewer though. Most would rather be coding or whatever their real job is, so yes they read canned questions from a textbook. In a way, bad interviewers may be a sign that the work itself is far more interesting!

Still, the approach has its advantages. "Listen, I know you'd rather get back to doing whatever you were doing before this. Let me tell you why I'm a good candidate .... ". And then go on having "developer-to-developer" conversation, ask them about their work, talk about your own. Obviously only do this on the canned questions, not the actual technical questions, unless you've already aced a few tech questions and are getting bored. Unless it's a hugely bureaucratic organization, this will probably receive a good review.

Another thing I've noticed is that it allows the interviewer to lower their guard as well, so you can usually get some more real "what's great, what sucks about the job", and I've even scored on "what's the pay cap of this position" multiple times.

Oh, by all means, DO judge the company by the interviewer(s)! Those are likely to be your colleagues. Your managers. If you have no respect for them after the interview, chances that you'll like them on the job are slim.

[edit] Also, the interview process should tell you a little bit about what kinds of candidates make it past the interview, to become employees.

> This is obviously not good advice

No. It is good advice. It's very good advice if you want to find a place where you fit well. Unless you're starving and just need a job, any job, right now.

Allow me to explain: After some ups and downs in my career I resolved to be true to myself. In my case that means calling bullshit on ridiculous things. You know what happened? I found out really quickly where I would have been miserable, and I was totally surprised by some incredible groups that wanted someone just like me, they just didn't know it.

Actual responses from me in interviews that found me a great fit:

Q: <contrived hypothetical completely unanswerable>

A: I don't know that sounds like a contrived question. I'd be asking why you have that problem in the first place.

Q: "people who do what you do are often vague and noncommittal"

A: that's because most people who do what I do haven't bothered to read the foundational texts, thus they rely upon tropes and ceremony. This field is full of muppets.

Q: what's your greatest weakness

A: sometimes I'm just a little too awesome smirk that says "are we really playing this game?"

Q: are you <blah blah> certified?

A: no I've purposefully avoided <blah blah> certification.I think it's heavy handed bullshit and I don't want to be associated with it or asked to do it

No joke these were real responses that got me great jobs and client engagements. They also quickly helped me discover people I wouldn't enjoy working with.

Your answers sound as if you have a strong feeling of superiority, avoid sharing something about you so as not to become vulnerable, and judge things heavily which you haven't experienced firsthand. I could be entirely wrong, but that's what your comment conveys to me. Does this only apply to interviews for you, because you despise them?

I can't say I've done what he did. But I do have felt the need to reply something like that to interviewers sometimes. Maybe now I'll do it, though I wouldn't burn bridges like he does, regardless of whether I'd feel comfortable working for them or not. You just don't know what the future holds..

I guess if you see my responses as burning bridges then you of course wouldn't want to do that. I don't see it as burning bridges at all. I don't take a nasty tone, but a more playful one. The point is to set the boundaries for what our relationship might be like. I have had companies come back to me months later after they figure out that they'd like a relationship on those terms because whatever they were doing in the past wasn't serving them.

Strong fences make good neighbors, yada yada.

No, I love interviews. I can be very extroverted. I'm charming and funny and a little brash at times.

I think it's critically important to explore, at the outset, what the boundaries of your relationship will be with any employer or business partner.

So for me, that means I lay down boundaries that I'm not interested in playing along with ridiculous ceremonies just because that's the way things have always been done. I think the way most interviews are conducted are ridiculous. Thus I push back and see what happens. You surely have your boundaries too. I feel your should respect them.

Show off strong outer armor and a soft, furry underbelly – I find this does well for me in interviews, as well!

I answered "alcohol and hard drugs" once to the famed "what's your greatest weakness" question. The guy looked at me totally shocked and I said "just f'ing with you, donuts hands down"

What was his reaction after the donuts comment?

And did you ultimately receive the offer?

I did get offered the job - I deliver answers to absurd interview question more jovial than it comes across in text...

He asked me seriously and I said something to the effect of "I eat at least one donut a day for the past five years despite saying 'this is the year I get beach abs' so if you want an honest answer I think you've got it"

> Where do you see yourself in five years:

You know, in my 20+ years of experience, there isn't a single moment I could go back to where I could even fathom what I was going to be doing five years after that.

Once in an interview with the CEO of a growing startup I got the "Where do you see yourself in 5 years?" question, I answered "I will be the CEO of this company", I got the job.

Of course, 5 years later life is completely different, that question is useless in my opinion, especially when the company itself is no older than 5 years or barely has more than one-year runway.

> I have just reached a point in my life where I will not be made to dance to the whims of the interviewer,

I think this interview style sends a signal, a lot of requirements that a programmer has to implement at work are made up ad-hoc/in an arbitrary manner. the message is that one is not supposed to question requirements too much and 'just do it'. (at least the filter is good enough to filter out a person who questions things)

That's lightyears apart from the interview bullshit.

On the job questions come with a context, or if not with someone who can offer context, or at least some guidance. (In the worst case you can ask your boss WTF to do. And maybe you'll get the answer, I don't know, and it's not important, it's up to you, etc.)

But never does the question what should this thing we're building right now will do in 5 years comes up without the answer, huh, good questions, but we have absolutely no idea, so we need more data and a shorter time-frame, and let's call up Gartner, Gordon Moore, Gardner and Tetlock, and maybe a few fortune tellers while we're at it too!

The problem is not the question, it's a fine question to start a discussion in the right context. (Maybe even during an interview. But it's not a "Q&A with tweetable answers" type question.)

In many cases the person conducting the interview is just a manager who was pulled in to help screen candidates. They don't have a lot of experience interviewing, so they resort to common questions like those in your example. It doesn't mean it's a bad place to work.

I make sure to remain polite, and the tone isn't aggressive. But the fact is any answer I give for the "5 years" question is not going to be the truth. So I attempt to refocus on the actual job; if that's not possible, that's OK. But it beats doing that whole song and dance.

I'm getting to be the same way. I've been fortunate enough to have found work in the last 15 years through friends, so the interview process is chatting at lunch as a formality.

One of the interview questions I've used in the past is "reverse a linked list".

Indeed, you (almost) never need to reverse linked lists in practice - but you often need to chase references of one kind or another (database, pointers, etc.) and do some manipulations on them that would result in a different list. If you have 20 data points, it doesn't matter what you do, but if you have 100M or 10B points, it makes a great deal of difference whether you do O(n), O(n log n) or O(n^2) or O(n!).

I think this is a good question, because it is an abstract version of the kind of problems that any non trivial code has to address. It is easy to describe, and easy to solve if you know what you are doing. I don't use it anymore because it's too common to be useful.

And yet, through the years I've gotten the feedback[0] that it is "far detached from real work", "tests how good I did in school rather than how good I am" and various other comments -- and almost always from candidates who did poorly; I've never gotten this feedback from anyone I would consider competent.

[0] I try to get feedback through whoever referred the candidate, whether it's the friend-of-acquaintance, or the recruiter. I don't, in general, trust direct feedback from someone I turned down (or hired, for that matter).

> I think this is a good question, because it is an abstract version of the kind of problems that any non trivial code has to address. It is easy to describe, and easy to solve if you know what you are doing. I don't use it anymore because it's too common to be useful.

This is precisely the problem the OP is talking about. When's the last time you ever did anything as low level as reversing a linked list? It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.

> And yet, through the years I've gotten the feedback[0] that it is "far detached from real work", "tests how good I did in school rather than how good I am" and various other comments -- and almost always from candidates who did poorly; I've never gotten this feedback from anyone I would consider competent.

Major red flag. I don't mean to be rude, but this sentence comes off as very arrogant.

> I don't, in general, trust direct feedback from someone I turned down (or hired, for that matter).

Why? Because they're in a lower caste? If you can't handle direct feedback, you're a terrible manager.

> When's the last time you ever did anything as low level as reversing a linked list? It's a solved problem, and something an employee would NEVER do in your company

I think that misses the point. A linked list is a simple data structure that needs little explanation, and yet uses indirection in a way that can get you into trouble. As such, it is a great toy problem to asses a person's ability to work with linked data. Reversing a list, is similarly, very simple to explain, and a quite rudimentary operation.

So if I'm trying to asses basic ability to manipulate data in code, it seems much simpler than trying to give them a real-world linked data structure and have them do a real-world manipulation on it.

The real problem is the assumption that this is a 'solved problem', i.e. that a) there is an one-true-way algorithm out there, and b) that is what the interviewer wants you to know (i.e. know in the sense of knowing a fact). That is true of too many interviewers, and that is the problem.

I can imagine setting that as a question in an interview. And I wouldn't be interested in 'the algorithm', I'd want to see that someone could figure it out easily. If someone came in and clearly had just learnt it, it would tell me nothing. I would immediately ask them something else to take them out of their rote prep. But if someone came in and couldn't figure it out, why would I think they'd be any better on 'unsolved problems'? In terms of being able to manipulate data, it isn't much beyond Fizzbuzz.

The 'it's a solved problem' attitude smacks of leftpad thinking to me.

Exactly. As an interviewer I don't get any data if you have an algorithm memorized and throw it on the board. I'm trying to assess critical thinking and problem-solving skills, I've voted to hire many people who clearly had never seen the problem before, didn't get a working solution in the time we had, but demonstrated great problem-solving approach.

> The real problem is the assumption that this is a 'solved problem', i.e. that a) there is an one-true-way algorithm out there, and b) that is what the interviewer wants you to know (i.e. know in the sense of knowing a fact). That is true of too many interviewers, and that is the problem.

These "solved problems" are the result of countless hours of research by computer science experts with masters and Ph.D degrees. For those of you who studied computer science in college, you likely had lectures where the teacher or TA walked you through the process of resolving each of these algorithms.

Unless you are hiring for companies like Netflix, Cloudflare, Google, etc. where you basically need to invent new fields of research in computer science, expecting a programmer to "come up" with the same solution in 10 minutes a team of CS experts solved after many hours of research might be a little unrealistic.

> These "solved problems" are the result of countless hours of research by computer science experts with masters and Ph.D degrees.

We're talking about reversing a linked list, right?

This is the second time I've had this kind of surreal conversation on HN (the first was summing a list of integers), and I confess I am completely baffled. We are talking about reversing a linked list.

I absolutely expect somebody with a even minimal programming competence to be able to perform simple manipulations on interconnected data. And I can't imagine a much simpler manipulation, taking into account the connections, than that.

Knowing the difference between a problem that you can solve intuitively, And one that requires hours of PhD research, is also a good skill to have.

Yes, it's only a linked list, and reversing it is trivial. The problem comes with the inevitable "OK, now how would you make it more space/time efficient?" question. I've had it countless times, and it's always infuriating because the answer is out there, on google, written by some PHD ages ago. The interviewer himself has read it, which is why he knows, and yet now he expects me to come up with something comparable in a "clean room" situation, usually involving a white board.

Really, those kinds of questions such as algorithm optimization only test your ability to come up with creative solutions in a stressful and time constrained situation. And even then you're not testing that because even the best of us will experience temporary brain paralysis in stressful situations, so you're actually just testing for luck.

And if that's what the working environment is like, fine so long as the candidate knows it's always like that and accepts those conditions (I wouldn't). But if that's NOT what the working environment is like, why are you even testing for it?

Really? Reversing a linked list? It really doesn't take a PhD to do that in O(N) time and O(1) space. I don't remember 'reading' an algorithm, isn't it just obvious that you walk through the list flipping the direction of the links?

Now of course, I'm making a specific point about a specific question. I'm not making a point about all 'algorithm whiteboard' interviews. Maybe you misunderstood the point, and thought I was. But I agree, many of them are utterly bonkers. Nobody is going to intuitively derive Floyd Warshall's algorithm with a whiteboard in 10 minutes. And coding trivia interviews are always pointless. If you can learn the right answers to 'pass' my interview, I'm not doing a very good job, imho.

But for very simple things like basic manipulation of the the simplest possible data structure involving a reference, I absolutely think a minimally competent programmer should be able to demonstrate their ability. Can you reverse it? Can you rotate it? Can you split it into buckets? Can you twizzle pairs? Can you insert in the middle? If you can't do those things with a cons-cell, in O(N), I can't imagine how you would cope with basic manipulations on more complex data.

Where I work, I deal with call processing (the stuff that happens when a call is made between two phones). If I were doing an interview, I might ask "We now have to deal with SIP, which means dealing with SIP messages, which are text based. We use C and C++ in this department, but we're open to other languages depending upon ease of integration. Which languages and libraries would you use to parse SIP messages, and why?"

It's a problem we face (or rather, faced and solved [1]) unlike reversing a linked list (or implementing a Red/Black or AVL tree or sorting data).

[1] Lua (very easy to integrate with C and C++) and LPeg (for parsing text).

TBH, that depends HIGHLY on the context. If I'm not familiar with Lua, do you not hire me? I'd answer "flex and bison" because they're (for me) extremely easy to work with, rest on solid theoretical foundations, have great tools for checking & understanding you grammars, and produce blazingly fast parsers - probably orders of magnitude faster than whatever you do in Lua.

Oh, and the parser code is C, so it integrates perfectly.

I would accept that (maybe with a follow up question if the code generated by flex and bison is re-entrant; I know that the older lex and yacc aren't). I also would accept "I would first look for an existing library that can parse SIP messages in C or C++." I might seriously question you if you answered "I would write code to parse SIP messages in C or C++".

And for the record, flex and bison might be faster than Lua and LPeg, but so far, it hasn't been an issue.

> if the code generated by flex and bison is re-entrant

This has been fixed long ago, you can definitely get reentrant parsers if you need to. Also, I don't know the SIP grammar, but fwiw, I'm able to write an extremely clean recursive descent parser in C++ (1); for simple grammars, it's often an overkill to go the flex/bison route. Also by "writing code to parse messages in C++" someone familiar with boost might be thinking about using boost::spirit. That's not necessarily a bad choice.

(1) Just looked - it's in fact too complex for that; but the point is, the right tools are dependent on the skillset of the person who answers, which may be different from your own/ your team's. Just because you find writing parser by hand daunting, doesn't mean that I do (I taught compiler design at the university, worked on a commercial C++ compiler, wrote many parsers).

Your interview question is good - but the key to it is the "why" part, not the answer of "how would you do it". And it's simply a different sort of question - your question tests the mindset + domain knowledge; The "reverse a linked list" question aims at filtering-out the developers that are not worth loosing time with. A developer that can't reverse a linked list is completely useless in a C++ project (in fact, his contribution is likely to be negative - doing more damage than adding value).

Yeah, dont worry about him. I fully agree with everything you are saying.

> Unless you are hiring for companies like Netflix, Cloudflare, Google, etc. where you basically need to invent new fields of research in computer science

Hmm. Scratch Cloudflare from that list. We are not 'inventing new fields of research'. We are looking for smart people of all sorts of backgrounds to come work on our stack.

I almost exclusively give candidates realistic tasks (slightly modified real world tasks) and give them under realistic conditions (free access to google, candidate's own laptop preferably).

It perplexes me. What do you gain by not doing that?

Time and (therefore, because interview time is limited) thoroughness.

A simple task on a linked list takes two minutes to explain and five minutes to solve. The kind of task someone might do for me day-to-day have a whole bunch of context, and require knowledge or explanation of architecture and data configuration. I don't want to spend more than a couple of minutes explaining the problem, just to test whether they can program their way out of a paper bag.

>A simple task on a linked list takes two minutes to explain and five minutes to solve. The kind of task someone might do for me day-to-day have a whole bunch of context

>The kind of task someone might do for me day-to-day have a whole bunch of context, and require knowledge or explanation of architecture and data configuration.

Did you think I wouldn't have to face this problem too?

If you can't decouple one relevant chunk of code from your software sufficiently to be able to give it with a minimum of context to an interviewee then I'd have serious questions about the level of coupling in your codebase and, by extension, your programming ability.

> Did you think I wouldn't have to face this problem too?

Yes you would. You would not faced that problem on the first day of job and you would be expected to take time to learn during first days/weeks of the job (depending on complexity).

Also, once you will have that context and understand architecture and data configuration, the task will boil down to "reverse the damm list" or "write one epic sql query" or alternatively "read this epic sql query and refactor it away for christ sake".

Thank you :D it always comes down to a pissing contest.

It's okay, we're very unlikely to work together. Feel free to keep doing it your way, and I'm not going to worry about a random HNer impugning my coding ability based on a comment about interviewing style.

No hard feelings. Honestly, I'd have a much harder time hiring if half the rest of the industry wasn't irrationally cargo culting Google's hiring practices.

Ah, I see, then I don't think you read the GP post, or I wasn't clear enough, since that is exactly a point I made.

Also, if you're only giving real-world work tasks, do you screen at all for basic competence first? I ask because I suspect we're talking about different things.

Also 2 (edit), does your work only involve one kind of task? How do you assess the breadth of their basic understanding without using simple problems? Are your programmers doing very repetitive stuff?

>Also, if you're only giving real-world work tasks, do you screen at all for basic competence first?

I've done both with and without. I honestly found it hard to make judgments about what effect what I was doing was having on the hiring funnel though.

If you're talking about that rather than later stage hiring then yes, I agree it's a different kettle of fish.

Whether you can give real-world questions depends a lot on the domain. If you're using popular open-source tools, you may be able to test a candidate on the ability to complete standard tasks with them. But what if you're using tons of proprietary software and working in a specialized domain where you expect people to learn on the job? That was the case where I used to work (quant finance). We need programming questions that assess candidate's general ability rather than anything domain-specific. The linked-list question is a perfectly fair non-domain-specific question.

>But what if you're using tons of proprietary software and working in a specialized domain where you expect people to learn on the job?

Then use it in the test! If you're judging people on how well they pick up proprietary software, give to them and make them do something with it.

My tests usually involve giving the candidate exposure to some software/module or other which they were not previously familiar and I judge them on how well they can work with it, applying general programming knowledge they will actually use day to do in the process.

But then, more likely than not, your testing how long it takes someone to learn your stack, and not how well they can solve problems once they're using your stack.

(or in other words, total time taken to do n tasks is mn + b, where m is the efficiency when using your stack, and b is the time taken to originally learn your stack. b is relatively large, so with n=1, that's a very different thing than with n=100.)

"But then, more likely than not, your testing how long it takes someone to learn your stack"

It tests how long it takes somebody to learn a small part of it, certainly.

Would that not be relevant for a job where you intend for them to learn your stack?

"not how well they can solve problems once they're using your stack."

A properly structured test could (and, clearly, both of our cases, should) accommodate both - to begin with, the ability to find their bearings in code with which they are not familiar and subsequently to solve problems within that context.

I usually do tests like this that last 45 minutes.

My edit more clearly explains the issue:

>or in other words, total time taken to do n tasks is mn + b, where m is the efficiency when using your stack, and b is the time taken to originally learn your stack. b is relatively large, so with n=1, that's a very different thing than with n=100.

For clarity, I work at google and I'd argue that for all but the most trivial problems, it would take even a good candidate more than 45 minutes to get their bearings. Most new hires do multiple days worth of codelabs before sitting in their desks.

In my experience the time it takes for programmers to get their bearings isn't necessarily about the triviality of the problem - it's usually about how decoupled/isolated the code you are working on is.

Sure, but my point is that it really doesn't matter when everything is unfamiliar. Imagine I put you into a situation where you're using piper, bazel, gflags, and protobuffs instead of git, make, argparse, and json, its going to take time to get your bearings no matter what. You'll have to figure out 2-3 new syntaxes.

Sure, you can limit scope, to things with 1-2 files where everything is handled for you and no data transfer between systems, building, or commiting required, but then we're getting into a class of problems where your limited to relatively simple, logical issues with very well defined APIs, a class of problems that data structures fit very, very well.

Research mathematicians are made in large part in the process of taking classes with tests for several years. Why don't they just ask the students to write papers if that is the end goal?

The tests, homeworks etc. are largely production of papers, on a lower scale perhaps, and interspersed with notes, but as far as I know not multiple choice nor mere reciting definitions or trivial algorithmic execution as most of the stuff is not concrete. I'm not a maths student so my maths tests are of the latter variant.

> But if someone came in and couldn't figure it out, why would I think they'd be any better on 'unsolved problems'?

It's an easy problem for sure, and I yet I could imagine many a competent programmer to lock up for reasons that have nothing to do with the problem and everything to do with the situation and sense of pressure they might experience during it.

I haven't worked on shrinkwrap software since 2006. For the last eleven years, every developer role I've had required me to be on call and troubleshoot when prod fails in the middle of the night. If you can't be relied on to do that, there are only certain teams that can use you, and I don't conduct their interviews.

The stress from a interview is vastly different from the pressure of fixing a prod issue.

It's like saying standup comedy prepares you for 3AM calls.

Wow, it appears my point was completely missed (and I was otherwise painted as a bigot), so let me try to explain better.

> It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.

Actually, just a couple of weeks ago we had something almost equivalent, that involved chasing "pointers" through databases and files. It was for less than 1M cases, but still - any O(n^2) solution would have been infeasible (and an nlogn solution about 20 times slower than an O(n) solution; in this case, it was a one-off, and the difference would have been between 1hr of runtime and 1day of runtime).

> When's the last time you ever did anything as low level as reversing a linked list?

Please re-read what I wrote. I wrote, in fact, "you (almost) never need to reverse linked lists in practice". I also explained that it is an abstract version of a kind of thing that DOES happen often.

> Major red flag. I don't mean to be rude, but this sentence comes off as very arrogant.

Perhaps. For all you know, I'm a dog, but I'll say that I'm soon entering my 4th decade in the industry, I'm well respected, I've hired (and fired) tens of people through the years (and interviewed hundreds if not thousands); and I'm well respected among my peers for having very accurate evaluations of people's abilities. I do not aim to be politically correct.

> Why? Because they're in a lower caste? If you can't handle direct feedback, you're a terrible manager.

No, because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now. I meant to say "direct immediate feedback", which is perhaps better wording.

After I know a person, I ask them (and usually get) the most direct and raw feedback I can get.

> it was a one-off, and the difference would have been between 1hr of runtime and 1day of runtime).

What's wrong with 1h vs 1d for a one-off thing? Your effective delivery time would be close enough to equivalent after all the administrative BS and customer expectations, and a naive approach piss simple to verify.

If it was going to mean the difference between 1h and multiple days, here's what I'd do:

- Fire up my trusty browser.

- Navigate to google.com

- Type in "reverse a linked list"

- Evaluate all of the solutions I find and pick the best efficiency / verifiability match to solve the problem in a reasonable amount of time.

Are you going to test that in an interview question? No, you're going to effectively "whiteboard" me, asking me to reinvent the wheel on the spot. And I'm going to resent you for it.

> but I'll say that I'm soon entering my 4th decade in the industry, I'm well respected, I've hired (and fired) tens of people through the years (and interviewed hundreds if not thousands); and I'm well respected among my peers for having very accurate evaluations of people's abilities. I do not aim to be politically correct.

And I'm in my third decade in the industry, all the respect, blah, blah, half a million lines of open source in the wild. So what? You can make the same mistakes over and over in any industry and never think to question if you're doing it wrong. This is ESPECIALLY problematic when it comes to people skills.

> No, because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now.

It's the PERFECT time to ask them, because it's fresh in their mind. A skilled interrogator can get information despite the subject's frame of mind. I'm actually surprised that you assume everyone you accept or reject is either going to be a sycophant or hold a grudge.

You WANT them to talk while the frustrations they encountered are still fresh in their minds. That's the most raw feedback you'll ever get.

You really do need to start treating them like human beings.

Wanna know how I evaluated potential game devs in person? I asked them what dev environment they were most comfortable in, gave them a computer hooked up to the internet and a picture of a tweetie bird, and said "Make the bird jump. Use google to look shit up." Worked amazingly well.

"If it was going to mean the difference between 1h and multiple days, here's what I'd do: - Fire up my trusty browser. - Navigate to google.com - Type in "reverse a linked list" - Evaluate all of the solutions I find and pick the best efficiency / verifiability match to solve the problem in a reasonable amount of time."

And now we're getting into absurd arguments. End of interview. Thank you for your time, but I don't think this will be a good fit.

> Are you going to test that in an interview question? No, you're going to effectively "whiteboard" me, asking me to reinvent the wheel on the spot. And I'm going to resent you for it.

Unfortunately, the real world business problem that needs to be solved is not "reverse a linked list", but rather "discover the relationship between the widgez and the widgix that the customer is after". That relationship, once you understand the problem domain, can be discovered by (doing something quite but not exactly like) reversing a linked list. And unfortunately for you, a google search for "discover the relationship between the widgez and the widgix" yields nothing. If you can't (do something similar to) reverse a linked list, you can't deliver the solution.

And it's fine you are going to resent me. I do not want to employ anyone who thinks "reversing a linked list" is an inappropriate question in a technical interview.

> What's wrong with 1h vs 1d for a one-off thing?

I was just giving this for completeness - in this case, it doesn't matter. But if it is 1h vs multiple days, or it happens often (say, every 3 hours), then picking the correct solution -- which your method will NOT generate -- is actually important.

> It's the PERFECT time to ask them, because it's fresh in their mind. A skilled interrogator can get information despite the subject's frame of mind. I'm actually surprised that you assume everyone you accept or reject is either going to be a sycophant or hold a grudge.

I ask again, have you hired people? because the "skilled interrogator" mindset makes me think you haven't. I've just negotiated with a person, whom I've had less than 2 hours to know. If I don't want to strike a deal with them, I want to be fair and tell that to them as soon as possible, at which point the vast majority of them just want to move on (and I happily oblige the very few who do want to continue the discussion). If I do want to strike a deal with them, we start the salary negotiation phase, at which point (assuming they are interested) they have a vested interest in me offering them as much as possible; which some people express by trying to please. These things are often subconscious.

I do not assume anyone is a sycophant or holds a grudge. But I do assume, at that point in time, that they are not exactly objective; and I cannot, in fact, calibrate their response since I don't really know them.

> Wanna know how I evaluated potential game devs in person? I asked them what dev environment they were most comfortable in, gave them a computer hooked up to the internet and a picture of a tweetie bird, and said "Make the bird jump. Use google to look shit up." Worked amazingly well.

Sometimes, you can do that. Often, you can't. Especially if the person has no experience in the field they are interviewing in.

"Here's a stock exchange API, write a bot that makes markets" is super simple for someone from the field, and would take hours (perhaps days) of grokking for someone who isn't. And making a bird jump would not necessarily tell me someone is apt for writing trading software.

> Unfortunately, the real world business problem that needs to be solved is not "reverse a linked list", but rather "discover the relationship between the widgez and the widgix that the customer is after". That relationship, once you understand the problem domain, can be discovered by (doing something quite but not exactly like) reversing a linked list. And unfortunately for you, a google search for "discover the relationship between the widgez and the widgix" yields nothing. If you can't (do something similar to) reverse a linked list, you can't deliver the solution.

How does testing a candidate on how to reverse a linked list help evaluate his ability to understand that the implementation is an appropriate solution for a given problem?

It doesn't. A different question will give me indication about that.

What reversing a linked list does tell me is, once the solution IS arrived it, that this person can implement it. And I repeat again, I treat this as an abstract version; real world problems often have solutions with many corners and are seldom if ever "Eureka! i just have to reverse the list".

Another real world problem I just recall is analogous in many ways to a "git merge" which requires tracing through a huge DAG. In these kinds of things, even if you realize that it's a git-merge, it is often not possible to generate a git repository just so you can run git-merge and observe the result.

There are two independent skills: abstracting a problem, and solving an abstract problem. I test for both but the linked list question only addresses the latter.

> There are two independent skills: abstracting a problem, and solving an abstract problem. I test for both but the linked list question only addresses the latter.

The problem (heh) with this is that, unless one is actively doing research on improving an existing technology or method, all abstract problems that might come up are already solved.

Every solution is one search in google away (including your git example) so there's no point to it. Paraphrasing someone else: if the time spent in coding a problem is expressed as mx + c, m is the time spent in figuring out what the problem is and c is integrating a solution. That c tends to be insignificantly small.

Frankly, as someone holding a degree, if you were to interview me and asked me to reverse a linked list, I'd do it, laugh in your face, and walk out.

On that point, interestingly, in your earlier examples of people you got feedback from, you didn't mention the most significant one, the one more likely to point out the problems in your process and that would give you actually actionable feedback: the people who rejected the job offer.

> the people who rejected the job offer.

Didn't happen many times, I did ask for feedback, but rarely got any; mostly along "your interview was fair, but I prefer a different offer I received". Which is what I would expect - unlike what some people in this thread expect, the candidate is maximizing their revenue and future benefits; saying anything nontrivial after rejecting the offer is not helping towards either, and potentially harmful.

> Frankly, as someone holding a degree, if you were to interview me and asked me to reverse a linked list, I'd do it, laugh in your face, and walk out.

That's good, actually. Means we have a culture mismatch, identified early. Win win.

> unlike what some people in this thread expect, the candidate is maximizing their revenue and future benefits; saying anything nontrivial after rejecting the offer is not helping towards either, and potentially harmful.

I had already noticed a very strong reality distortion field in your other comments[0] but this one is just weird. I have been a candidate, most people in this thread have, exactly why their saying what they'd do is not a reflection of what a candidate might do? For instance, I cannot conceive why saying "anything nontrivial" matters in regards to revenue and future benefits. I've done it before (probably not "laughing in their face" but, then again, I have yet to meet an interviewer with such a clear misplaced arrogance) and so far, rejecting a company and giving feedback has not hurt me in any way; they have no impact on my future because I rejected them.

For that matter, how did this even address what I said? I see not how it isn't a non-sequitur, other than you believing that you have a strong enough pull in the industry that you could badmouth me into not getting a job if I rejected your offer and told you it was because your interview was terrible. Of course, that is not the case.

> That's good, actually. Means we have a culture mismatch, identified early. Win win.

Yeah, no, that wouldn't have been deduced from that interview; one could get a culture mismatch by exposing the applicant to the actual company during the process. What I said would only bring to light that your interview process is terrible and doesn't actually account for people who aren't desperate for getting the job (which, in itself, explains why you think everyone you hire is a sycophant, they probably have to).

Of course, I understand your inability to see that it might be a problem with you specifically; no, it must be that the interviewee has a problem with "the culture" of the company.

[0]: For instance, there was no majority saying that the creator of homebrew "was in the wrong" regarding his tweet. At most, there was an even split, although there was far more complaining about interview processes like yours.

> other than you believing that you have a strong enough pull in the industry that you could badmouth me into not getting a job if I rejected your offer and told you it was because your interview was terrible. Of course, that is not the case.

It is obvious our different experiences give us different views of the world. I would just add that I have seen this happen, mostly in business settings. A lot of people take things personally.

I don't particularly take things personally myself; it's all business or math to me. But when I was younger, I used to give brutally honest feedback about everything, and there were a couple of investment opportunities that it cost me (and I cannot say that it created others). I learned to play the politics game as I matured. I hate the game, yes, but I needed to start playing to break a glass ceiling.

> Yeah, no, that wouldn't have been deduced from that interview;

No, it would have surfaced immediately. I try to hire people whose attitude is more of the "yes, I can do anything that's needed. But why?" rather than "I laugh in your general direction since I think you are wrong".

> with you specifically;

Me and google. And facebook. And hundreds of other companies. You are entitled to your opinion that e.g. Google's culture is the problem.

Which is not to say my company is as successful as google. Just that I share the idea that culture fit is important. I recommend reading "Peopleware" if you haven't yet -- they have some empirical anecdata for that.

Also, I'm amused at how my "I prefer to ignore noisy data" comment (which is all my comment said, explained several times) was taken to mean "I think everyone's a sycophant or crybaby". I think it reflects on the readers' ability to consider that I'm calibrating my stats differently than they expect, or rather lack of it. And I calibrate it that way based on decades of interviewing -- I didn't start that way.

> I ask again, have you hired people? because the "skilled interrogator" mindset makes me think you haven't.

Wow... Just wow... You've let a big part of your personality slip out with that one.

> And making a bird jump would not necessarily tell me someone is apt for writing trading software.

And now we're getting into absurd arguments.

You believe that a technical interview ends with skilled interrogation of frustrations and I'm being absurd.

>This is precisely the problem the OP is talking about. When's the last time you ever did anything as low level as reversing a linked list? It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.

The thing about such things is I know I can do it. But maybe it'll take 5 minutes, or maybe an hour. Maybe I'd attempt to do it differently given an unfamiliar scenario for which to be doing this in the first place; maybe I've never done this specific task in this specific language, and I'll take some extra minutes figuring out how pointers or references work in the language I'm using in the test. Maybe I've recently enjoyed some momentum on high level concepts (i.e. I produced software that fulfills a business need) and it'll take me a while to refocus my attention on low level implementations.

Engaging one's brain carries some very variable overheads. If you just give me 5 minutes and watch me write every single line of code and comment about it while I'm doing it, then it's just not going to happen.

You guys do realize that reversing a linked list is not 'solving a difficult programming puzzle'? Seeing the number of applicants that can't read lines from a file (in any given language), I'd say this question is adequate if you want to employ someone with programming experience.

You're right about that particular exercise. I kind of used it to make a more general point.

Indeed. For the record, I let people take however much time they need, if they get stuck I ask them to discuss their approach, etc. I'm not looking for the solution (as parent said, it is a solved problem), I'm looking for evidence of problem solving skills. And you can demonstrate them, even for this question, without actually being able to write down a working solution. (I don't want to go into everything here, but one of the things I look for is people who consider the case of an empty list - critically thinking about special cases is an important trait)

That said, work environment often has some kind of time constraint. If one can't solve any of the (simple, abstract, representative) problems that I pose within a 2hour time frame, it is possible that they are not a good match. And unfortunately, I rarely have the leisure to hire someone for a month to prove otherwise.

[from a comment of yours below]

> You're right about that particular exercise. I kind of used it to make a more general point.

That may be true for others, but my interview questions are all of this mold: abstract, with a short and simple (but not trick-required-here) solution, close to (an abstract version of) a real world "business problem".

I agree that technical interview questions (much like many aspects of life) put the wrong kind of stress on some and puts them at a disadvantage. But, and I always ask this[0], what alternative do you suggest?

[0] The answers so far have been essentially either "no idea", "let me prove to you on my terms" (which, when broken down, is rarely economical for either side or at all possible), or "just trust me on this"; I find none of them satisfying.

Then I'd say you have some pretty decent justifications for your approach. I'd take the negative responses you got as constructive rather than inflammatory: if it made you review your justifications and think again about whether approach is fair, then they did their job.

In many ways, writing on the Internet is much like the interviews we are discussing, especially in a place such as this, which has so many highly opinionated experts. I get why you got hostile responses to your post, but I also get what the reason for writing that post was. In person, the exact same words would not have lead to such a high-strung feeling as one gets when posting about anything controversial on the Internet.

On the Internet, if you go out on a limb, someone will always cut the branch.

Yep, you have the "far detached from real work" reaction.

Plus, if you consider "you" (or an imaginary candidate you project yourself upon) risk not being able to answer it in 10 minutes because of such the high pressure it would put on you, then it basically means that you did not even try to answer it in your head, or that you were not able to.

But this is the moral equivalent of asking a person who pretends to be a mathematician to do a 3 by 3 figures multiplication. It won't show much if they answer correctly (at least demonstrate a correct method to do it, never mind potential mistakes), but you can easily dismiss them if they fail.

Now programming is a little bit different, because I admit there can be some positions where people can somehow contribute despite being incapable of lots of basic things. It depends on several factors, though, and even then I'm not sure the end result would ever be very solid. But I understand that in some contexts, such people would not be hired...

Im not sure I agree with the premise that interview questions like that that may never come up in the job are useless. To me it doesnt matter. They are easily solvable and any competent programmer should be able to solve them.

Of course I think more relevant questions are obviously better.

> When's the last time you ever did anything as low level as reversing a linked list? It's a solved problem, and something an employee would NEVER do in your company, let alone search for the optimal solution in 10 stressful minutes while you breathe down his neck.

I need occasional semi ad-hoc datastructures and manipulations like that. The requirement is never "reverse the list", but it is often something like "this api here needs input in opposite order and data happen to be in list". I would not see reverse the list as something specially unusual.

> When's the last time you ever did anything as low level as reversing a linked list?

It's a topic covered in a sophomore-level data structures course over just a few days. If a college sophomore can pick it up that quickly (and "sophomoric" is a word for a reason), it should be no problem for a seasoned programmer to (re-)learn it in a couple of hours.

How about in 10 minutes at a whiteboard while the interviewer waits patiently to judge you?

> I don't, in general, trust direct feedback from someone I turned down (or hired, for that matter).

That's a problematic attitude, in my view. It reads to me that you think you have all the answers already, or at the very least that you view yourself as being superior to people you interview. That lack of humility is troublesome for a number of reasons.

As far as your interview question, do you normally not interview candidates to develop software? Your question is appropriate for a grunt who's entire responsibility is to thoughtlessly code up somebody else's design. It's not appropriate for someone expected to contribute thoughtfully to the engineering of a software product.

> That's a problematic attitude, in my view. It reads to me that you think you have all the answers already, or at the very least that you view yourself as being superior to people you interview. That lack of humility is troublesome for a number of reasons.

I would correct that (if I could still edit) to "direct immediate feedback". What I meant was, if someone I just turned down for a job says "it's stupid", I don't consider that useful data; similarly, if someone I just hired says "This is the best interview question ever", I don't consider that useful data either.

I do ask for, and usually get, direct and raw feedback from people I work with (both those who report to me and those I report to).

> As far as your interview question, do you normally not interview candidates to develop software? Your question is appropriate for a grunt who's entire responsibility is to thoughtlessly code up somebody else's design. It's not appropriate for someone expected to contribute thoughtfully to the engineering of a software product.

Can you elaborate on what you consider "contributing to the engineering of a software product"? and more generally what "software engineering" entails?

I mentioned as explicitly as I can that I do not expect anyone to reverse linked lists. However, it is an abstract question that is not unlike the kind of things that arise in practice, e.g. figure out inverse relationships in data. And gives a good ground for discussion about things like requirements (time, space, correctness, verifiability, boundary conditions, erroneous inputs, etc) -- inside a well defined and (supposedly) familiar context; The same problem in a business setting would require a lot more definitions and would be frighteningly foreign for some people (to digest within an interview).

How do you interview?

> Can you elaborate on what you consider "contributing to the engineering of a software product"? and more generally what "software engineering" entails?

Actually, not in a space appropriate for an HN comment. Here's a very brief summary: I think in its current state there is very little actual engineering in this industry. It seems to me to consist of a large number of folks who approach every problem like a CSX01 project. It could move toward a more mature engineering footing if there were less focus on CS trivia and more on broader, deeper understanding of systems and relationships between them.

To be blunt, your "reverse a linked list" question is something I consider an example of the Law of the Instrument (https://en.wikipedia.org/wiki/Law_of_the_instrument): every problem is a CS nail to hit with a CS hammer. It's great for exploring the items you note at a very low, narrowly-scoped, and shallow level. It's not so good for gauging actual problem solving ability or engineering ability.

> How do you interview?

Interviewing is a crap shoot, and my methods are at least as flawed as any others'.

Phone screens with a shared editor are the place I ask questions like yours, except I'm not looking for a correct implementation, complete solution or discussion about it: I'm looking to see if they at least have some vague notion of what writing source code entails (even if it's just because they read about it and memorized it).

At the in-person interview I tend to do two things: discuss in depth a project in the person's past and a higher level discussion about something pertinent to their work at the company. For me the latter has varied from details for some feature to be implemented in a real time system to motivating a data exploration approach in the context of developing a data science product.

> I think in its current state there is very little actual engineering in this industry.

Agreed.

> It's not so good for gauging actual problem solving ability or engineering ability.

Also agree. It is not the only question I ask. However, I only want to hire people who, when the need arises, can actually use a hammer. I am perplexed that people find that offensive.

> At the in-person interview I tend to do two things: discuss in depth a project in the person's past and a higher level discussion about something pertinent to their work at the company. For me the latter has varied from details for some feature to be implemented in a real time system to motivating a data exploration approach in the context of developing a data science product.

Agree. However, I also do some technical parts in person/

> Indeed, you (almost) never need to reverse linked lists in practice

And there are thousands of other things that you (almost) never have to do, yet you do, throught the course of your career. When they arise you're not an expert in them, but after some careful research you become one, or become enough of one to say which solutions are better than others for the problem you have. And you'll be able to implement it, support it, and explain it to the people with whom you work.

These types of interviews don't see the forrest for the trees. The problems that you have had won't always be like the problems that you will have. What's important is the ability to solve problems you've never seen before (a graphql request took down the database cluster), not recite known but not widely remembered tasks (balance a b-tree in-place).

Getting to the point where you have to start to worry about Big O is the hard part, not the other way around. And Jesus, I really hope we work in places where if you're unsure about Big O you can ask someone and they'll help you, not judge you.

Edited to add: What these types of interviews say to me is the companies who use them have no interest in their employees growth and education.

> What's important is the ability to solve problems you've never seen before

Most definitely! And that's exactly why I no longer use this question (as I wrote in my original post) -- by now, everyone has seen it.

> Getting to the point where you have to start to worry about Big O is the hard part, not the other way around. And Jesus, I really hope we work in places where if you're unsure about Big O you can ask someone and they'll help you, not judge you.

I disagree. I deal with lots of data (just ordered another 100TB storage the other day; I do that every couple of months these days). Toy solutions that work well for small 100GB datasets often don't extend. Without a good grasp of complexity, it is easy to build solutions that don't scale properly, or cost $100K more in hardware/AWS than they should. I've seen it happen a lot of times. As they say, "a year in the lab can save you a week in the library!". If you don't know that you should go to the library, you're likely to lose that year.

> What these types of interviews say to me is the companies who use them have no interest in their employees growth and education.

Some of us are bootstrapped in a cut-throat market and can't afford to plan employee retention 10 years into the future; And neither are most employees interested in that.

> And that's exactly why I no longer use this question

This matters? Any other question stinks just the same, like my example of balance a b-tree in-place.

> I disagree...

What is it you desagree with? "Getting to the point where you have to start to worry about Big O is the hard part, not the other way around." or "I really hope we work in places where if you're unsure about Big O you can ask someone and they'll help you, not judge you."

> Without a good grasp of complexity...

I don't see how the interview style you seem to advocate finds people who are good at understanding complexity. This comes from experience, the opportunities to have worked on unfamiliar projects, not from reading a how to pass a job interview book.

> Some of us are bootstrapped in a cut-throat market

Some of us work in toxic environments that are hostile to women and people of color but ¯\_(ツ)_/¯, right?

> can't afford to plan employee retention 10 years into the future; And neither are most employees interested in that.

I would wager that everyone is interested in learning something new or deepening an existing skill in whatver job they take no matter the time period. The alternative would be to, which you seem to advocate, take a job, stand still in it for a few years, then wehn you're bored or feel like you're becoming obsolete, read some books on nights and weekends on whatever new trends people ask about in interviews, take a new job, stand still in it for a few years, and so on.

> What is it you desagree with?

"Getting to the point where you have to start to worry about Big O is the hard part, not the other way around."

Or maybe I misunderstood what you meant. I think that unless you are doing mostly "do this and then do that" style programming (e.g. CRUD), you have to be aware of big-O, or you're going to hit a lot of walls. In things I do, anyway.

> I don't see how the interview style you seem to advocate finds people who are good at understanding complexity.

Complexity here being the well defined CS term about space and time, of which big-O is the customary notation: How your resources (time, space, etc) grow with the problem size. I suppose you meant complexity as some measure of cohesion/details/size - in which case, you are correct - this kind of questions do not address it (others in my interview do, though I admit it is much harder to judge one's view of other-complexity in an interview).

> Some of us work in toxic environments that are hostile to women and people of color but ¯\_(ツ)_/¯, right?

Some of us make unrelated remarks, but ¯\_(ツ)_/¯, right? What exactly were you trying to say here?

> I would wager that everyone is interested in learning something new or deepening an existing skill in whatver job they take no matter the time period.

Then you would lose this wager. There are so many people who just want to clock in, do their job, clock out and get paid. Yes, even in technology. And you know what? that's fine; They are not lesser people because of that - they likely care more about hobbies or other things, and would happily quit their job and have fun if they won the lottery.

> The alternative would be to, which you seem to advocate, take a job, stand still in it for a few years, then wehn you're bored or feel like you're becoming obsolete, read some books on nights and weekends on whatever new trends people ask about in interviews, take a new job, stand still in it for a few years, and so on.

Not at all -- but there's a huge difference between "being interested in something new" and "train someone in a new field they are not proficient in" especially when there is no commitment (nor could there be) on their side. The latter costs money to the employer, which some can afford and some cannot -- either way, it is not a basic human right.

> you have to be aware of big-O, or you're going to hit a lot of walls.

Of course you do, along with many other things - many other things that you won't know you need to be aware of because you haven't come across them yet or because they haven't been invented yet. What I said is that it's important to know how to figure these things out, not be able to recite them from memory.

> What exactly were you trying to say here?

You said "Some of us are bootstrapped in a cut-throat market and can't afford to plan employee retention 10 years into the future" To me this is a defeatist attitude. Our jobs are fundamentally about solving problems, and this, like my example, has a solution.

> Then you would lose this wager.

You can't have it both ways. You seemed to say that your preferred interview style selects the best of the brightest, but now it selects people who just want to clock in and out?

> Our jobs are fundamentally about solving problems, and this, like my example, has a solution.

Unfortunately, if resources are limited, you have to compromise. Google's resources are limitless, mine isn't. if your solution is "close up shop and go work for someone who has more resources" then I'm not the defeatist. What's your solution for constrained resources?

> You can't have it both ways

I do try to select for the best and the brightest. I pointed out that some people like to clock (your wording implied everyone likes to learn new things on their job). The best and the brightest who are not Jeff Dean tend to change jobs every few years even if they work for Google. And no, I cannot afford to employ Jeff Dean, I admit. Can you?

> What's your solution for constrained resources?

Which resources are constrained?

Our examples are 1) (yours) cultures that are hostile to employee growth and education, and 2) (mine) hostile to women and people of color. These are solvable problems, even ones as awful as Uber as we're seeing play out. As for what we can do, we can start to hold our industry accountable. When we hear about some unicorn at some absurd valuation, rather than simply applaud we need to ask them about their culture, how they treat their employees and customers, do they have a healthy work/life balance, what exactly are they doing to foster diversity, what's their impact on the environment, does their product actually make the world a better place, are they in service of the surveillance state, and so on. We need to redefine what it means to be successful in our industry.

> And no, I cannot afford to employ Jeff Dean, I admit. Can you?

How did we get here? I said "What these types of interviews say to me is the companies who use them have no interest in their employees growth and education." As I read the thread that follows from this point, it seems like you agree which is at odds with your defense of these types of interviews that started our discussion.

> Our examples are 1) (yours) cultures that are hostile to employee growth and education

You assert that, but I have no idea why you would think that. That's why I was perplexed at your statement. Can you elaborate? Non of the examples you give liek work/life/diversity/environment has featured into the discussion, nor the definition of "success in the industry"

> I said "What these types of interviews say to me is the companies who use them have no interest in their employees growth and education."

And I ask again, is, why would you assert that? It seems trivial to you, but I'm completely at loss. All I said was that I cannot, at hiring, offer a 10-year growth plan for a person. It does not mean I do not expect, encourage and help them to grow (quite the contrary).

> You assert that, but I have no idea why you would think that.

Why I would think what? That this is true or that you think so too? As for the latter, because you said so...

* "Some of us are bootstrapped in a cut-throat market and can't afford to plan employee retention 10 years into the future; And neither are most employees interested in that."

* "All I said was that I cannot, at hiring, offer a 10-year growth plan for a person."

* "There are so many people who just want to clock in, do their job, clock out and get paid."

* "The latter costs money to the employer, which some can afford and some cannot"

All of which seems to say to me that a person better come with all of the skills that they could ever possibly need now and in the future because they're not about to get any support from their employer or teammates. Just sit down, shut up, and mash on that keyboard. What you seem to advocate as growth and education is something that a person should do own their own time and at their own expense. People who aren't willing to attend meetups, read research papers, work on side projects, etc. after hours are just a bunch of clock punchers who go through life without contributing much. This sounds pretty horrible to me.

"10 years" and "the costs" are just rhetorical attempts to set the bar so high to say that it's just not possible or too unreasonable to care about lifting up your teammates. This is easily provable false simply because of the companies that do this already no matter the tenure of their employees and at very little cost.

As for the former, I say there are far more important skills that an interview should focus on rather than if someone can recite the pros and cons of different sorting algorithms and their Big O costs because this is easily teachable. But when you don't care about growth and education, or just simply teamwork and collaboration, you want to know if candidates already have the answers to the problems they'll encounter, which, of course, is totally ridiculous to me.

> I don't, in general, trust direct feedback from someone I turned down (or hired, for that matter).

Would you please share where you're working so we can avoid applying?

(I understand the sentiment, and I deserve it for the wording that I cannot edit now -- but, copying myself from other posts:

because I've just hired them, or just didn't hire them; either way, I think it is irresponsible to trust the statement of a person I don't actually know and who may either hold a grudge or want to please me right now. I meant to say "direct immediate feedback", which is perhaps better wording.

After I know a person, I ask them (and usually get) the most direct and raw feedback I can get.

You miss the point. Everyone has something interesting to teach you. Listen to the sages and the sycophants alike because there's no profit in closing yourself off.

But in this case we have all the people who didn't make it to the NFL complaining about the NFL combine, and how in actual football they don't have to bench 225 pounds.

> I've never gotten this feedback from anyone I would consider competent.

You may not have gotten it personally, but some very smart people have given precisely that feedback regarding those sorts of questions.

https://twitter.com/mxcl/status/608682016205344768

On phone now, so can't search but ... there were a couple of threads on HN when that tweet was made, and IIRC the majority actually didn't think he was in the right.

Being smart and being a good fit are not equivalent. Linus Pauling was definitely smart - and most definitely wrong about quite a few things (most famous was his statement that "there are no quasi crystals, only quasi scientists", a derogatory remark about Dan Shechtman who later got a nobel prize for those quadicrystals)

The tweet isn't a counterargument that disproves your premise; it's a data point you seem to be unaware of. Certainly there are smart people who think your question is fine, but if you think that the people who disagree are mostly incompetent, I think that's a defense mechanism.

Put it this way: if I decided someone was incompetent as a programmer based on some criteria, and then I later found out that that person wrote Homebrew, I would take a pretty hard look at the criteria, and how strongly I believed in its predictions.

With all due respect for Max Howell, his response is a defense mechanism. "invert a binary tree" is an ill defined question (maybe he was joking, I don't know), perhaps he didn't even understand what he was asked and didn't bother inquiring. He was not told WHY he wasn't accepted - it might have been for displaying a "rock star better than though" attitude which some places reward but Google does not.

I do not disqualify people who cannot reverse a list (as long as they demonstrate critical thinking skills and reasonable capability). I do disqualify people who get up in arms like Howell did, and I dare say this tweet does show Howell to be a bad match for many teams, regardless of his technical skills; He is likely not a competent team player. (As much as can be inferred from the tweetstorm that happened 2 years ago, and as much as I can remember it).

Howell's actual competence or the particulars of his interview are out of our scope. His tweet could have been a petulant defense mechanism or it could have been eminently justified; we have no way of knowing. (And if we're paying him due respect, let's leave out speculation about rock-star attitudes or his competence as a team player entirely please.)

All I'm talking about is the predictive power of the "please rejigger this data structure" interview question, and how much faith people place in it. I mean, if someone looks at the Howell thing and says, "well, any interview can have false negatives, but questions like this are still good predictors of competence", I'd certainly have no argument with that.

But a lot of people looked at it and said, "Nope, if he failed a question like that then he's not a competent programmer, regardless of what he's built". When someone says that, they're implicitly claiming that "passed an interview question" is a better predictor of programmer competence than "built and maintained one of the world's more popular developer tools". Doesn't that strike you as a pretty extreme amount of faith to put in an interview question?

> But a lot of people looked at it and said, "Nope, if he failed a question like that then he's not a competent programmer, regardless of what he's built".

The responses I remember (memory bias possible) were along the lines of "well, he might have built that but is not a good match for google if he failed a question like that".

I have actually calibrated my interview in the past on friends, colleagues, employees and students, and I know that it has predictive power for the kind of work I do (which does NOT include, e.g. CRUD or the simple if-then-query logic which is often referred to as "business logic").

With respect to Max Howell - I will, indeed, leave speculation out, but will say that the attitude he displayed on twitter would disqualify him from a lot of places, regardless of his technical merits. (And as for the real reason he was disqualified - it was pure speculation on his side).

The response that stuck in my memory was from Jonathan Blow (whom I greatly admire, apart from the following), and amounted to "then you're not a very good programmer".

Personally, I strongly suspect that the predictive power of interview questions depends ~90% on the knowledge and experience of the interviewer, and the specific questions aren't that important. I reckon the disconnect here is that you and JBlow are judging things as the guy had failed the interview question as you would have administered it. And it's possible that that's roughly the case, but it's also possible that he failed the interview as administered by a bad interviewer - who, say, cut him off after the first trivial error rather than suggesting he check his answer for bugs, or whatever.

As such, I take no issue with data structure questions per se, I just find it hard to buy this idea that they're massively predictive - compared to, say, having shipped lots of good code for a long time.

> With respect to Max Howell - I will, indeed, leave speculation out..

This would read better if it wasn't followed by speculating where the guy's qualified to work (based on memories of a tweet from three years ago!) and then speculating what he does and doesn't know about an interview he was in.

> This would read better if it wasn't followed by speculating where the guy's qualified to work (based on memories of a tweet from three years ago!) and then speculating what he does and doesn't know about an interview he was in.

The process at Google (of which I'm not intimately familiar with, I admit, but I have read a lot about) is such that if you are rejected, you are very rarely told why (I know of one case in about 30, and it was an offer that was rescinded based on some bureaucratic reasons). It might have been the reason, for sure; but it is extremely unlikely that this is what he was told, thus I infer speculation on his side. By accounts from Googlers I know (and some who commented on those threads at the time), it should be considered truism, not speculation, that he was not told the reason.

Also, I'm not speculating about where the guy is qualified to work. I was referring to several places I know for sure (some high profile, mostly in finance) where such a rant on Twitter is enough to disqualify you (and get you fired if you're already employed). I don't know Yelp specifically, but it seems like one of those places[0]

[0] http://fortune.com/2016/02/22/yelp-employee-ceo/

> It might have been the reason, for sure; but it is extremely unlikely that this is what he was told, thus I infer speculation on his side.

Nobody's suggesting HR sent him a letter - presumably he formed his conclusion based on what was said in the interviews. It may be that he did so unreasonably, as a defense mechanism, and it may be that whatever was said left little doubt why he wouldn't be getting an offer. All we know is that he felt there was enough information to form a conclusion, and the principle of charity compels us to assume that such may be the case.

Honestly, I'm not trying to pick an argument or anything here. I just think your confidence judging this guy's character and hirability and so on is way out of whack considering that it's based on a three year-old recollection of a tweet.

I'm astonished how you've been treated in this thread. You've actually mananged to convince me that reversing a linked list is a good interview question!

Because it's considered very rude by normal people.

For a hypothetical example:

I've spent a day for your interview. I aced it. I was told there is somebody better. I was then asked to waste more time in form or feedback.

Same goes for references. In fact, from a real life example:

My company needed an entry level technical writer. I referred them this kid with a fresh comp sci degree, eager for a job. Reviewing code: yay, he thought he made it. Got turned down because they wanted only a rockstar entry level, technical writer. Then chose to not get one.

So, given everything, because I'm a polite person, I'd rather not answer than tell them they are delusional and rude.

I think this is a great question and tests basic programming knowledge. It's similar to the parallel parking portion of our state's driving test. Most people don't parallel park enough for it to be a factor in daily use but it's a good way to test overall knowledge and control of the vehicle. If you can parallel park, it's a good indicator that you can do other things safely as well.

From your submissions you seem to be keen on sorting algorithms and some low level programming, so I'd say you are somewhat biased on what programming skills are important, while I wouldn't even necessarily disagree with it, depending on the specifics of the job to interview for.

My replies to dodge the question would be: That's not a question, that's a command.

> I don't use it anymore because it's too common to be useful.

I've found that having common questions used across many interviews allows for more objective direct comparisons. I like this one in particular because even if they solve it quickly, you can move on to questions about cycles in the input.

What do you use now?

It was never my only question.. I vary through the years.

Another favourite of mine is basically "implement a very sparse 100Mx100M matrix". Thought process much more important than result. (important things to consider: access time, storage size, static vs. dynamic, use cases, adversarial coordinate selection).

I actually consider the "cycles in the input" question a bit of a trick question -- I calibrated the test on people I knew were good, and no one who didn't know it already was able to come up with "the trick" in reasonable-settings-for-interview-time, and not many were able to come up with any O(1)space O(n)time solution.

The discussion with those who don't know the solution is useful, of course.

> The discussion with those who don't know the solution is useful, of course.

Agreed, the discussion on these questions is the whole point, not whether you can come up with a "correct" solution (and I tell candidates this up front).

I generally draw a small example input with a cycle, and ask what their list-reversing solution will do with it, which I feel makes it much less of a trick question. "Can you work out what the code you just wrote will do with inputs that don't meet the assumptions you made?" seems like a fair thing to ask anyone who will be required to write code that handles untrusted data.

By linked list, do you mean a doubly-linked list?

Wouldn't that make it (even more) trivial, since you can just traverse from the tail backwards? Assuming it has a tail pointer, of course.

Yeah, I suppose so. I only asked because reversing a singly-linked list just seems like a PitA.

Reversing a singly-linked list is actually easier: you take each element off the front of the old list and put it onto the front of the new list.

I thought this guy was being sarcastic?

Only from the experience of good people interviewing at bad companies.

Interviewing bad people at good companies goes more like this:

Q: Can you explain the difference between a noun and an adverb?

A: I've worked at the UN for 20 years! I'm an accredited translator! I translated for Putin and Obama!

<sigh>

As an interviewer, sometimes I feel a little silly having to ask a candidate super basic things like what the runtime complexity is for inserting an item into a list or array. But then a third of candidates that I interview fail pretty badly. At his last fortune 500 job one of my coworkers started every interview with fizzbuzz and had about a 50% pass rate on it. Maybe that's a problem with candidate sourcing, but there's a lot of people out there that can't code.

There is a core truth that there are "programmers" out there who cannot program, and the only way to filter them out is to have them implement some algorithm live.

But I think a lot of the grief around whiteboard interviews comes from all the additional crap that's grown up around that, including:

1. That ideally a candidate should code up solutions on a whiteboard, while standing. That it's unnecessary, or even bad, to try to replicate a working situation by having the candidate seated at a computer.

2. That there's a correlation between difficulty of question and quality of developer. So a candidate who can implement both fizzbuzz and quicksort at whiteboard from memory would be a better developer at your company than one that can do fizzbuzz, but not quicksort.

3. That the ideal programming question has some sort of "trick". So the ideal flow when a candidate is answering a question starts with the candidate implementing a naive solution. After which the interviewer says "it's possible to do better". Then the candidate ponders for, ideally 30-60 seconds, whereupon they realize the trick and code up the optimal solution.

My ideal interview process recognizes the need for a developer to demonstrate the ability to code while at the same time recognizing the above interview style for the cargo cult exercise that it is.

What are these companies that ask fizzbuzz? Every company I have applied to lately, asks me impossibly tough questions on hackerrank, which have to be done in like 30 minutes each. That would mean perfect syntax, and perfect coverage of test cases with no knowledge of what test cases are failing, or else I am rejected.

EDIT : Add to that, perfect time complexity.

I had a question recently, which was so tough, that forget the allotted time, I have asked my friends about how to solve it and they seem to have no clue either.

Can you link to it / describe it?

This is all I remember now :

An array represents a binary tree like this :

is represented as : 1 2 3 * 4 5 * 6

so apart from the last level, any missing child nodes are represented as *. You need to find the set of unconnected nodes whose sum is maximum. Child-parent relationship means they are connected.

What is the runtime complexity for inserting an item into a list or array?

O(n) - regardless linked structure or array backed.

How is inserting an item at the head of a linked list O(n)?

Easy:

    insert(list, node):
        node.next = list.head;
        list.head = node;
        sleep(list.count++);

Remove the `sleep` in second version, and quote significant speedup. See also http://thedailywtf.com/articles/The-Speedup-Loop

It's an average. Sometimes you insert at 0, sometimes at 1, sometimes at 2, ... sometimes at n. On average it's O(n).

In this case it's the worst-case performance.

Who said you insert at different locations?

Nobody. But who said you insert at the head? And who said the whole thing is list-backed? Insert is just insert--not a lot to work with there.

Performance complexity is spoken to general cases and averages unless indicated otherwise.

"insert" does not stipulate prepend or append. The latter two operations are O(1) both for linked and array backed [amortized O(1) for array backed and actually faster due to high constant costs of allocating/releasing/iterating linked structures]

O(n) means it takes less than or equal to linear time to preform the operation. So if an operation is O(1), it is also O(n).

O(1) is a subset of O(n).

Depends what you mean by insert, list and array.

there's no "time complexity" associated with inserting an item to an abstract linked list. The answer varies with implementation. Any answer <= O(n) could be correct.

"Linked list" usually refers to a very specific structure (regardless of language, or whether it is implemented as pointers or arrays or whatever) - one in which to find the middle element you have to traverse at least half the list.

Singly linked list or doubly linked lists are still considered linked lists. Skip lists, even though they technically are lists of linked items, are not generally considered a "linked list" data structure.

Even ignoring "implementation details", insertion at head/tail would be O(1). Why would you traverse the list to insert an item?

But average case or worst case are in the middle (for a doubly linked list) or the end (a singly linked list without a tail pointer).

There's no "worse case" for insertion if the api just says list.insert(item).

The half-trollish point being that nothing is "obvious". Indeed, list insertion is O(1) in popular programming languages s.a python or JS.

When I was doing Delphi, the first technical question was, "what is the difference between a function and a procedure?" That was a quick and easy way to figure out who had written a program in Delphi before. Anything more complex than that or FizzBuzz doesn't add any value.

Case in point: I interviewed a guy who nailed even the most obscure stuff, but he couldn't finish a project. He was great at small, complex problems, but he couldn't ship. We needed someone who can ship.

Having a hard time understanding why those candidates were even called in. No pre-screen coding practice? No pre-screen phone call?

I don't know what their exact process was, but I'd guess there was just a non-tech phone screen before the onsite. It's a tricky balance, some people get insulted if you ask them to complete a pre-screen coding exercise. "I have 15 years of experience, stop wasting my time!" On the other hand, I've interviewed a few people with that much experience, supposedly code-writing lead engineers, that couldn't write the code to insert an item into a sorted array.

There are a lot of people who have more credentials than skill, which isn't easily detected by looking at a resume.

Even more concrete example, something like:

Q: Explain the difference between they're, their, and there.

Q: Principle or Principal?

> Everything is in a flat structure, except when it comes to salary and responsibilities.

This part got a laugh out of me.

Otherwise, the analogy feels really stretched and at times feels straight up incorrect. For instance, I've never sat through a technical interview administered by non-technical staff. I've likewise never been quizzed on the history of computation.

I agree that the programming interview in the US can be overly algo-and-whiteboard happy, but I think this critique is unfair and possibly even outdated (my most recent round of interviews involved more live coding, and less whiteboarding, than when I last interviewed 4 years ago)

I had a technical phone screen done by an HR several times, or at least it felt like it was a completely clueless person. Once I could tell exactly why, when after he asked me what is complexity of binary search and I replied "O(log)" he asked what base? (for non-technical people reading: logarithms of every real base produce exactly the same O() result being different only by a constant factor).

It's been a while since I took a math class, but it seems like the logarithmic base is key to understanding what kind of problem you have and the type of solution needed. Comp Sci naturally uses 2, biology and finance tend towards e, etc.

Base 2 problems tend to require clever recursion and dynamic programming to solve, while the natural logarithm often requires calculus.

But then again, I'm not a mathematician - just what I took away from a broad undergrad.

Sure, if you are actually evaluating logarithm the base is very important. However, as I said, an entity designated as "O(log)" is exactly the same for every base so selecting a specific base here makes zero sense. Likewise, O(n^c) is as same as O() of any polynomial of power c, if somebody is writing O(3n^2 + 5n + 366) instead of O(n^2) - you know they don't quite understand what are they doing.

I've definitely had phone screens where the interviewer was non-technical and asked technical questions they didn't understand, and were just looking for form answers they could tick off. Even from Big 5 companies. But in-person technical interviews were always with currently practicing engineers.

Yeah I wasn't taking phone screens into account. I'd edit my comment to clarify that I was thinking of in-person interviews, but the timeout has elapsed.

I think I disregarded HR screens because the questions (IME, at least) are often so trivial the experience of the interviewer doesn't really matter. There have been a few times when I've disagreed with a screeners pre-defined answer, but none of those cases resulted in a rejection or a halt to the interview process.

> > Everything is in a flat structure, except when it comes to salary and responsibilities. > This part got a laugh out of me.

I had to smile when one interviewer described his company to me as very process oriented, but without the overhead.

Gayle Laakmann McDowell, author of Cracking the Coding Interview, commented on Facebook about this article (all text below is her words): https://www.facebook.com/groups/hackathonhackers/permalink/1...

> Weird analogy. Companies don't ask candidates the history of binary search trees, computer architecture, or anything like that.

A better analogy would be if they gave this translator a particularly challenging piece of text to translate -- for example, one that didn't have a clear right answer and the candidate had to discuss different tradeoffs.

But... then that doesn't seem like quite so silly of an interview process.

There are absolutely valid criticisms of whiteboard interviews, but most criticisms made are either based on terrible implementations of whiteboard interviews or based on stuff that's just incorrect. (Yes, it's totally fair to criticize a company who conducted a flawed whiteboard interview. But that criticism doesn't apply to the system as a whole. That same company could mess up whatever your favorite interview style is, too.)

> By the way: I don't actually know how translators are interviewed. But one of my best friends interviewed to be a journalist with some major New York newspapers (WSJ, etc).

She was already a journalist before this, so they had lots of public writing samples for her (analogy: GitHub code samples).

Did they just hire her based on this? Nope!

She had to do a live writing test (analogy: whiteboard coding interviews). She also had to do a pitch session to talk about different potential stories she could theoretically write about (analogy: design/architecture interviews). Plus some behavioral interviews.

Why not just look at her writing samples? Unlike for coders (which might not have public portfolios representing a significant portion of their work), basically all of her work product was actually public. So why not just hire from that?

Well, because all they see is the final output. They don't know what direction she was given, how long it took her, how much editing/collaboration was involved, etc. A crappy writer in a particular environment can churn out good work -- because someone else is doing a lot of the work. Looking at the final result is actually not a great measure of someone's skills.

Coding interviews aren't that special.

> A better analogy would be if they gave this translator a particularly challenging piece of text to translate -- for example, one that didn't have a clear right answer and the candidate had to discuss different tradeoffs.

What you describe is what interviewers think they're engaged in when they ask CS trivia in interview settings. They are not, and in fact are much closer to what the article describes than the picture you paint.

Entirely! Example: When has anyone ever actually needed to know the exact details of a red-black search tree?! In my decade of professional software development has this not appeared once. Not even remotely. Yet this is apparently "standard required knowledge" for interviewing at google, amazon and co.

The point the author is making is that interviews often question knowledge that is so far removed from actual work (aka the Arab influence on modern day Spanish), that it is kind of a joke.

I call this mind wanking. Interesting and may be fun. But not actually relevant.

Instead of asking far removed questions, I interview with actual current hard engineering problems that I or my team is faced with. That way you learn something, they know what actual work would look like. No time wasted. Win win.

> Yet this is apparently "standard required knowledge" for interviewing at google, amazon and co.

What makes you think that? My experience is that Google at least would never ask a question about red-black trees. I doubt Amazon would either. These big companies stick to questions that don't require a lot of specific knowledge because they know that a question about red-black trees mostly tests how recently you've studied red-black trees. I've done a bunch of whiteboard interviewing, and I would be shocked to be asked a question about red-black trees at a competently run company.

"Cracking the Coding Interview" explicitly mentions red-black trees as a topic that you're unlikely to see in an interview (for obvious reasons).

I think the places you see these sorts of questions asked are smaller companies that are imitating the big players without really understanding how to do it right.

First hand experience. I applied for a job at google and this was given as an example of knowledge that I should revise for the interview.

Was that a long time ago? I've heard stories about things like that a bunch of years ago, but my impression was that things have changed.

>> Was that a long time ago? I've heard stories about things like that a bunch of years ago, but my impression was that things have changed.

By then you have wasted a good candidate's time, who doesn't want to interview at your company anymore. Go hire the folks who memorized RB tree.

Was it before or during interview? If during, like "you should have studied it", then it is wrong way yo do it.

If it was before, then I think it is fine. It basically test whether you are able to learn something like red-black trees when needed. I would consider such question a good one.

Can confirm that red-black trees didn't come up once in my Amazon interviewing career.

>In my decade of professional software development has this not appeared once. Not even remotely.

I don't say this to personally attack you, but Google and other top tech companies have made it clear that years of experience no longer holds the same weight as other industries, perhaps because other industries are more tightly regulated with what its members can do. Otherwise reference checks would handle most of the technical interview ("Did this guy actually build X, Y, and Z at your company?" "Yes." "Great!"). You can spend 10 years in a bad position and Google can't know every company to decide whether it should hire from your company or not.

This has been one of software development's strengths: literally bootstrapping yourself from "Hello World" to 120k+. It's hacks and cheat codes compared to becoming a doctor or physical engineer (the ones that need a license). It means a disadvantage upbringing can be overcome with time and tenacity. It's a romantic notion, but because of this, you have to evaluate the candidate or risk a bad hire. And bad hires, as we all know from the same echo chamber that gave us teach-yourself-boostrapping developers, can utterly ruin your business and you never want to be a manager known for making a bad hire.

Otherwise, we'd all take an exam, refresh it every number of years, and be free to disregard most of the technical interview. Getting well-regarded regulation and certification is no easy task, and I suspect there's not movement behind this because you can't kill people with software as easily as you can with a surgeon's scapel or a bad bridge.

The industry is actively fighting against glue-things-together developers by asking serious CS questions. It hasn't accepted that we have to branch the "Developer" position into more well-defined roles and add specialties (Security, AI, UI/UX). It hasn't accepted that a lot of work is actually glue-things-together.

There are certain jobs where "knowing the details of a red-black search tree" are actually very important. These jobs are generally related to implementing low level, high-performance libraries in certain contexts, conducting research, etc.

They're appropriate questions for interviews in some cases. But very few. Even at the companies you note.

I'm a recent grad, I like to keep track of interview questions. Personally I've been asked dozens of whiteboard questions across a number of companies. I have friends who have interviewed at countless places. I don't know a single person who has been asked a red black tree question.

Where do these get asked.

I was asked a question like that once and knew the general details but was quizzed on the specifics - needless to say I did not get the job and my elementary knowledge about red-black trees was specifically mentioned as a reason for my rejection. This was at a highly publicised tech startup that people consider a very good employer in the industry.

What year?

This was this year, a couple months ago.

Apple, though that was one interviewer and a few years ago.

Edited OP to clarify that I did not write any of this.

What's annoying in tech is that even if you do have public work (E.g. one or more popular open source projects), you still have to pass the technical tests in order to get hired.

Nobody actually looks at your code and think "Oh wow, that code is clean, it works well, it's loosely coupled and high cohesion - We should hire this guy".

Often they just want you to be able to solve lots of tedious algorithmic problems REALLY QUICKLY. The horrible thing is that even if you CAN solve these problems, you might not be able to solve them all within the allocated time (unless you've practised a lot recently).

> Often they just want you to be able to solve lots of tedious algorithmic problems REALLY QUICKLY. The horrible thing is that even if you CAN solve these problems, you might not be able to solve them all within the allocated time (unless you've practised a lot recently).

More to the point, at zero times in my career have I needed to do fundamental algorithm implementation. If I did, I'd question what people were doing by reinventing the standard library.

And even if I did end up needing a particular algorithmy solution to something, I'd be reading the relevant parts of TAOCP, looking at prior art, discussing it with other people in a team environment...

None of which is implementing red-black trees on a whiteboard. Really. It's completely orthogonal to software development.

Don't worry about Gayle. She is very mistaken about the nature of software engg. It'll be funny if she had to interview for a real job and couldn't answer anything unrelated to whiteboard questions.

CTCI was pretty fun to work through, but the questions were extremely superficial and the answers were often quite flawed. It is disappointing to see major companies rely on that style of interviewing, especially for senior positions.

At Twitter I did a few hundred technical interviews and in general I don’t like asking CS student questions but you do need to figure out if the person can code. To this end I devised a whiteboard problem that actually mimiced a real problem. After I left I posted it on GitHub:

https://github.com/spullara/interviewcode

It is important when doing lots of interviews to have a question that you know well and can be used to benchmark across your interviews. Something relevant to the job is an added benefit.

When I was working at Ooyala I used to do the typical data-structure interviews (trees, graphs, etc), until one time an interviewee asked me: "Do you guys use all this in your day to day?" I felt ashamed to have to answer "not really", which I did.

After that, I started asking for answers to problems that I really had in the day to day. Some of them actually had real complexity (like doing binary search to overcome a buggy HTTP request items pagination library).

I think the main problem with data-structures and algorithms questions is that, the person that originally envisions a question, understands the "soul" of the question and how to vouch the candidate as she solves the question. However, when someone else takes that question to apply it, he only sees whether the answer is right or wrong, without looking at the resolution process.

>> I think the main problem with data-structures and algorithms questions is that, the person that originally envisions a question, understands the "soul" of the question and how to vouch the candidate as she solves the question. However, when someone else takes that question to apply it, he only sees whether the answer is right or wrong, without looking at the resolution process

In addition, I think that when someone asks a binary tree traversal problem to a candidate, the interviewer starts fretting over trivial edge cases, function signatures and coding style etc.

Whereas, if you would have asked a distilled version of that pagination problem, I would be so excited to have a chance to solve that problem and we could focus on my ability to solve that problem instead of my coding style on a whiteboard.

Forgot the part when you must have a public portfolio of work you did for free in the recent years even though the job in question will keep intellectual property under a lock.

Indeed-- require people who have a large public portfolio of translations they've done for free, and then have them sign a contract stating the company owns the rights to all translations they do while employed and for a year after they leave, regardless of whether they were on company time or not.

"translated early christian heresy text from greek to modern latin. Sadly won't be published ever by papal order"

I am tempted to ask to see the company's open source contributions

I find this article one-sided and too exaggerated.

In reality, the sad state is, in my opinion, confluence of the following forces:

1) HR people more often than not have barely any clue about the topic. They must, unfortunately, play this charade because if they knew the topic they wouldn't be working in HR.

2) The technical people who prepare the questions typically believe they are too busy to spend time thinking about the problem and instead decide to settle on any test. The assumption is that a good candidate will be able to navigate any kind of test better that a bad candidate. In view of the technical person, this is just a screening, the real purpose being to elliminate as many phonies as possible.

3) The candidates more often than not have highly exaggerated view of their abilities. Unfortunately, high demand means that the market reaches for lower and lower quality of "resource" leading to comical situations where a large portion of the workforce (especially in countries like India) is developing software by shuffling around keywords until the code compiles which entitles them to call themselves Senior Engineers. Real senior people have no problem finding a job to the frustration of others who find the situation "unfair" and the entire process "rigged".

One more note on the process: while the cost of failed interview for candidate is quite low, the cost of making a mistake is very substantial for the company.

> It means we do Agile. Everything is in a flat structure, except when it comes to salary and responsibilities.

Pure gold!

In the end we went with a different candidate. He doesn't speak the specific language we need, but he has experience with similar words in a different language and he's a lot cheaper.

And also, http://www.jasonbock.net/jb/News/Item/7c334037d1a9437d9fa650...

> Interviewer: But how many years of walnut do you have?

> Carpenter: Gosh, I really don't know -- was I supposed to be counting the walnut?

> Interviewer: Would you say you're an entry level walnut guy or a walnut guru?

Oh, hey, I just had that interview. It's called the California State Systems Software Specialist examination. Almost 60 questions like:

"Create processes (e.g., install, configure, maintain, secure, backup/recover, etc.) to ensure that technical staff are consistent with vendor documentation, application requirements, and departmental standards."

"Consult with internal/external entities regarding services provided by systems software teams and answer questions/inquiries in technical areas such as connectivity with departmental systems, data exchange, security, etc."

For each of which, you must answer:

Recency

a. Performed the task within the last 2 years

b. Performed the task within the last 4 years

c. Performed the tasks more than 5 years ago

d. Not performed

Years of experience

a. More than three years experience performing this task

b. More than one year to three years experience performing this task

c. Less than one year performing this task

d. Not performed

Level at which the task was performed

a. Supervised or served as an expert on task

b. Performed task as a lead

c. Worked independently on task

d. Worked under close direction/supervision on task

e. Assisted another person on task

f. Not performed

If you don't happen to have been assigned the proper tasks in recent years to score highly enough on the test (by whatever algorithm they use), then you are not eligible to even apply for any Software Engineer role in the state, and can't retake the test for 6 months.

Lol the average chippy on a building site wont have worked with much if any hardwood.

Having said at my first job our in house wood shop where capable of museum grade work in any wood you name but we where no 1 or 2 in the world for the areas we worked in

I thought it would be something along the lines of,

"Could you translate '¿Donde es el baño?'"

"I'm sorry, I don't do well on tests. I thought we were going to talk about my past projects on my résumé. In fact, I'm quite offended by your question. Good day, sir!"

"That should probably be '¿Dónde está el baño?', but the question is clearly asking 'Where's the restroom?'"

"Wrong, sorry. My sheet here says 'Where is the bath?'"

I keep saying this but no one seems to be listening. Use triplebyte, interviewing.io, or recurse.com for all recruitment. You are not going to beat those folks in terms of quality of candidates. They have data and stats. You have anecdotes.

Because it seems like a misc plug? Recruitment is ridiculously ridiculous. Data and stats, so what?

I'm not affiliated with any of them. The plug is genuine. I've been on both sides of interview table to know it's a crapshoot. You really are better off with one of those three than on your own.

Those companies go out of their way to reduce as much bias as possible. In the long run we are all better off if we rely on data instead of anecdotes and fancy resumes.

related discussion

Google: 90% of our engineers use the software you wrote (Homebrew), but...

https://news.ycombinator.com/item?id=9695102

Maybe he was an asshole? You only got one side to that story.

> Note to self: when we send a rejection letter to this candidate, be sure to include feedback that is too vague to be of any use to them, but will avoid any lawsuits.

Haha, what feedback?

What rejection letter? :)

> Yeah, you know, trans-comps, Translation Competitions. Those were you put a bunch of documents in Sanskrit, throw some beverages and pizza, and invite translators everywhere to lock themselves down for 48 hours to see who can translate those the fastest. Sometimes they add free prizes to spice things up. Have you ever won any of those?

lol... I actually won a trans-comp when I was in (middle?) school: I was attending a catholic school and we were taught latin and went to some translation competitions: you had to translate a chunk of text as fast and accurate as you can. It was fun :)

Heh, heh. My favorite bit is toward the end:

    ...we do Agile. Everything is
    in a flat structure, except
    when it comes to salary and
    responsibilities.

If only this were an exaggeration.

The issue is that most companies jumped aboard this insanity - it was originally meant for the top companies to get the best people, now every crappy company offering $40k salary is doing this...

Reminds me of those companies that throw online algorithmic puzzles to candidates. They do some good only to the companies running the puzzle services.

One time, I got one of those questions: two columns, on the left names of some people, on the right names of some frameworks, please associate who created which framework

I really should have left at that point: >1 hour lost there...

Do interviews really go like this at many companies?

Here in NYC I have never had unreasonable interviews even close to that. And I interviewed for a lot of senior developer positions and consultant positions.

In our own startup we have a completely different approach. Our motto is "People live lives. Companies build products."

We like to hire and work remotely because that eliminates geographic restrictions and lets people work asynchronously. We've found that the better the system for asynchronous communication, the better the long-term productivity and maintainability.

We use a common folder structure, code conventions, for each project. Developers build fully documented reusable components that are re-used across projects. Every developer is very replaceable (meaning our losses are limited if they leave or scale back their time). This is actually a great thing for developers given our compensation model (see below).

If a developer does something wrong (like checking in syntax error), we first check if this is something we should fix in the system (add a linter to the pre-commut hook). There are so many amazing open-source tools today. It's a compoundibg snowball to design a good system. Sometimes the COO job feels like an architect/developer, just like DevOps, but for people and configuring processes and systems instead of programs or servers.

The best talent is the one who already knows or can learn your platform (in our case https://qbix.com/platform) and become productive. Who is eager to grasp new SKILLS (like debugging javascript with asynchronous call stacks) and get familiar with useful TOOLS (like Google Chrome).

We hire from anywhere and prefer to work over the internet. Even our compensation model is different than what most companies do - it aims to attract independent people and entire teams, and compensate them based on the contributions they actually do. We want to grow a snowball in a transparent way, and motivate people by giving them ownership of a product or feature instead of focusing on making them sell their time as full-time employees who commute to an office.

I'd love to get feedback on the compensation model btw: https://qbix.com/blog

The article was fun to read but the job of an engineer cannot be compared to a job of a translator.

And regarding "full-package" translators I think that web developers should be able to write both frontend and backend part of an application. It is not that difficult to learn. Programming is not something you can learn once and then repeat the same actions for the rest of your life.

I see folks around here constantly complaining about this issue (not incorrectly) but is anyone actually doing anything about it?

I mean, we can whine all day and remind each other that it really does suck, but that does little to address the problem.

> Please stop saying trans-comps.

Oh, I can only hope...

Reply on news.ycombinator.com