How to interview engineers (2020)
spakhm.com
spakhm.com
Despite what the author says, this type of interview is definitely something you can train for. If it’s real, and successful the author has essentially created a test for people who have spent a lot of time in competitive programming, or who spent a lot of time on interview prep.
He's created a test for people like this or really smart people and in fact both tend to be fairly good hires.
If they're smart, they're smart, and if not they've demonstrated high conscientiousness (the psych term for being organized and hard working).
I'm not saying 8 hour leetcode interviews are a good idea, but one challenging toy problem like this gives decent signal that the candidate is at least smart or hard working (maybe both) which is more than many candidates.
That's what programmers do, that's what you should check if they can do. Not if you can one liner some assembly to write hash into db that will trigger an event in main application all in one go.
I've given hundreds of coding interviews. I always timed the candidate (with their knowledge). So I have a general idea of the distribution of programming speed, and my guess for this particular problem is to solve it in 10 minutes, you need both talent and practice... just one isn't enough. So asking people to solve it in 10 minutes ends up filtering out untalented people who have practiced a lot (and talented people who haven't practiced as much)
Rigid interviewing techniques like that can easily weed out great candidates. Your confidence in your own process leads me to believe you may have missed a few great ones yourself.
Max gets brought up in these discussions a lot, but honestly I think the no hire was a good decision from what I know. Homebrew was a triumph of product design more than technical prowess, I suspect Google might have hired him as a product manager. Max was by his own admission was not a great programmer[0] but interviewed solely for a software role. I don't have a CS degree either, but I wouldn't hire someone to work google scale software if they couldn't invert a binary tree.
https://www.quora.com/Whats-the-logic-behind-Google-rejectin...
One idea here is to give candidates an open problem in CS and ask them to solve it. Which is a nice idea, but one of the other things I learned through administering hundreds of interviews is that open-ended questions aren't great interview problems because they depend a lot on creativity/lateral thinking/insight, which highly benefits from being in a relaxed frame of mind. So these questions end up being a test of how relaxed the candidate is.
Stress creates tunnel vision, which is terrible for generating interesting research ideas but it's OK for solving these kind of leetcode problems. So checking for a very high level of programming aptitude, as a proxy for CS research ability, is an approach which is a bit more fair to candidates who are stressed out by interviews. (I also think it is a pretty decent proxy, because doing great CS research requires you to quickly & fluidly generate & evaluate algorithms / data structures which might solve your problem, which is a big part of what a great leetcoder does.)
If you're targeting a very high level of generic programming aptitude, it is arguably most fair to make use of a standard method of measuring it. Leetcode problems are the industry standard for measuring programming aptitude. People know to practice them a lot and they know what to expect. If you came up with your own unique way to measure programming aptitude, that would create an even greater burden on candidates to practice (they'd have to do a whole different sort of practice in order to succeed in your interview), and also create anxiety due to an unexpected interview format.
I think a lot of readers are overreacting because they didn't pay enough attention to this bit: "It's applicable if you're building an extraordinary team at a hard technology startup." The vast majority of companies in SV are not doing hard technology and don't need an extraordinary team. People should not feel inadequate if they aren't capable of improving the state of the art in technical areas of CS such as databases. This is ordinarily the domain of PhDs, and filtering for demonstrated algorithmic aptitude (as opposed to academic credentials) is actually a pretty egalitarian approach.
> "It's applicable if you're building an extraordinary team at a hard technology startup."
And you're accepting that a timed question about tic tac toe is enough to prove you're capable of being on an "extraordinary team" at a "hard technology startup"?
Really?
If you don't agree with me that the fluency I described is a significant asset to advancing the state of the art in CS, what do you think a significant asset is?
Sure. Again. I don't really think an algorithm that shows up as an introduction to algorithm proves much more than a person read "Intro to Algorithms". So again, a timed introductory problem proves some elite technical skill?
> If you don't agree with me that the fluency I described is a significant asset to advancing the state of the art in CS
You're building a cute lil strawman. I think the question is totally out of line with the stated goal. If a college sophomore can answer a question, you're not really assessing much of anything. Also, working at a "hard startup" has nothing to do with "advancing the state of the art in CS".
> what do you think a significant asset is?
If I'm handling hiring for a "hard startup" and am in search of engineers fit for an "extraordinary team", I'm probably going to spend more time finding applicable skills than opening up to Chapter 1 in the closest algorithms book.
In my observation the default state of a student reading a textbook is it goes in one ear and out the other. Most students temporarily acquire a superficial understanding of the concepts which allows them to answer test questions and get a decent grade. To see something in the wild and instantly recognize that it's isomorphic to a concept you studied years ago requires a level of mastery/passion well beyond what it takes to get an A. (I'm talking about school in general here, of course the fact that interviews index so heavily on data structures/algorithms ends up distorting things a lot from the baseline. Still, if you solve this problem in 10 minutes you're one helluva sophomore.)
>Also, working at a "hard startup" has nothing to do with "advancing the state of the art in CS".
I think of "hard technology" like rethinkdb as being exactly equivalent to cutting edge stuff that advances the state of the art in some way... again, maybe there's just been a misunderstanding/miscommunication here
I don't know what to make of it.
Also in the "not sure if this is bad satire or just bad" camp
In my opinion that is complete BS (as is much of the rest of the article) - the candidate could have seen the problem or something similar before or used a particular technique or data structure that suits the problem - while that may indicate a general technical ability it could have been something they spend weeks figuring out last month but can now roll out again quickly.
Never base hiring decisions on a single task or question. Even if they did dreadfully on what you think is a basic topic that everyone should know, don't automatically disqualify them. Mark them down but ask other questions.
For everyone’s questions about satire, the blog is called “zero credibility”
These interview questions exist only for the purpose of filtering out as many as possible from an enormous number of candidates, not to find the most suitable engineers.
However, the fact is these people are inundated with job offers and very generous swag - like laptops. Everybody wants to hire them. It's hard to compete for one of these candidates let alone fill an engineering team. Centering hiring around that is an act few can follow.
Founded RethinkDB? So is this about how to run a tech company into the ground?
Everything else comes third, fourth, etc...
I can write code that runs, but is messy, uses short variables, and generally is one big blob and doesn’t handle expansion very well or I can spend time to consider future needs and write sensible variable names.
Not sure which is correct.
In smaller shops it’s going to be more hit and miss because you don’t know the level of the candidates you’re up against, but probably style points, comments, and good structure will be weighed more highly.
Read this and everything else in the voice of Edna Mode from The Incredibles, and it makes sense.
Gross.
> If you decide you want to hire the candidate, the interview must last at least six hours (with an hour break for lunch). Have your engineers interview the person, one by one, for about 45 minutes to an hour each.
Gross. The hardest pass.
> First, the candidate needs to feel they've earned the privilege to work at your company.
Gross. Equally hard pass.
> Second, your engineers need to feel they know the measure of whoever they're going to be working with.
Gross. Random engineers aren't qualified to assess talent or fit. Random engineers _may be_ qualified to pick candidates that match their gender/race biases though.
> and more importantly spend time having a little trepidation about how they did.
I would go so far as to say "Fuck you" to this author.
When I was fresh out of college the 'obvious' approach just appeared, and I was off to write it.
Now, any problem I am slow to peel apart in my mind, decide what the best way to approach it is, based on the language I'm using, what I'm feeling right now, readability, maintainability, etc.
Tic Tac Toe? Interesting; should I store board state as a 2d array, or a single array? The obvious approach would be to try and play every possible game with backtracking, but perhaps instead it would be simpler to just generate every possible permutation of 5 Xs and 4 Os? Would that work; on the one hand a badly playing O might lose after just 3 Xs, but we could still fill out the board if we 'kept playing', so it maps one to one, so maybe! Of course, then you risk situations where both X and O wins, so maybe not? Also, what about rotations; we could reduce our work by ensuring we didn't try to solve for cases that are just rotations of one another, but is the book keeping of that more work than just brute forcing it, given the constrained nature of the problem? Etc etc.
Writing working code matters, yes. But if you're looking for people who think just a few seconds before typing anything, and then seem frustrated they can't type fast enough, you're optimizing for juniors.
I hope it's all satire.
This would not include all possible end game states. A value is either X or O or empty. It's possible to end with empties.
xox --- ---
corresponds to two different games depending on where you placed the first x, so no
The entire point is to hire people that 1) are interested in exploring algorithms on their own time (i.e. they're interested in more than just getting paid for a job, they're interested in interesting problems too) 2) are competitive and want to win 3) are hard working and intelligent (i.e. they had to study for these competitions and do well in them).
You might miss a lot of smart people but it's pretty likely if you ask really hard algorithm questions that you'll filter out any not smart people. If you're reacting badly to this style of interview, that's because you're not their target talent pool.
Not so. From the article:
> First, I cannot myself pass this interview. Last time I tried, I got the correct answer after about forty minutes or so. I could get it down with practice, but it doesn't matter-- I think slower than I type. That's a no hire. The point of the interview is to hire extremely talented engineers, not engineers as talented as me.
There's _some_ truth in some things that he says though, even though HN doesn't appreciate it. Like, anecdotally, I have a friend who refused a Google offer (when Google was much smaller but still a big-ish name) and went for a startup because the interview problems at the startup were very difficult and he thought they were going to have interesting problems to solve. I took note of that and made sure my interviews were as difficult as the candidate could take it - like, go progressively harder until the candidate is stuck, then back off. This works fairly well especially with just-out-of-university hires, there's a certain type of people that notice it and like being challenged. And they're often very good employees. (of course, what the author suggests in the article is WAY over the top, I'd agree his process is broken).
Serious question... how did these kind of fifth grade playground insults become normalized on HN? It's a blatant violation of the guidelines https://news.ycombinator.com/newsguidelines.html and yet somehow comments like this get upvoted.
The author of this post is either ignorant or malicious. Some of their behavior is so ignorant or malicious that it warrants direct adversarialism. This blog post is gross.
Curious, what about my post included a "playground insult"?
Would you please not post like this to HN? It's not what this site is for. No matter how strongly you disagree with someone about $topic, it's not ok to dump poison into the ecosystem.
If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
I'm sure you can make your substantive points thoughtfully, so please do that instead.
Edit: corporate bullshit aside, this is a great way to find good engineers. The manipulative tactics and social standards don't sit well.