One Hour Hire - Hacking Engineer Interviews
rocketship.instacart.com
rocketship.instacart.com
They seem to be forgetting that potential engineers are interviewing them just as much as they're interviewing the engineers -- and such a crazy up-front time commitment is a sure way to weed out the engineers who have better things to do with their time.
This may be scalable for the company, but it certainly isn't scalable for engineers looking for jobs at multiple companies.
We've automated the technical skills part of it. Hiring at Instacart isn't a completely automated, no personal interaction, process.
In between the candidate applying and the technical challenges is usually a phone call where we talk to the candidate to make sure we're on the same page. We find out what their goals are and if we can help. We explain how we work, and see if they'll like it. We explain the hiring process. We answer questions they have about us.
If we're both happy at the end of the call then we move them into the automated process.
"We are serious about who we hire, so we're careful about who we do hire. However, we realise you're interviewing us as much as we are you, so we're equally as serious about showing respect for your time. We really appreciate it, so we'll pay you $m/hour while you complete this tests. We think these three tests should take a competent engineer around n hours to complete, so we'll pay you a max of $m*n for completing the process.
Good luck!"
As the OP highlighted hiring is an expensive process (expensive engineers interviewing candidates, double whammy of them not working on product while interviewing), it could be a lot cheaper to just automate some of the process, but make it more humane by compensating the interviewee.
There is a definite balance to strike between the humanity and scalability aspects, but I think it's an interesting idea.
Unfortunately, it wreaked havoc with the candidate's schedule, especially if they had another job, and took too long to get to a pass/fail for them.
The current process let's the candidate work on a production problem, but at a time of their choosing and on their own schedule. It also gets feedback to them much sooner, within a half hour or 3 hours.
Only something along the commitment of the 0.5-1hr "weed out" question can/should be done remotely.
If you're going to require people to do ~7 hrs of tests, that had better be at the office, pairing, with other folks (and at least give them a free breakfast/lunch/dinner!). 7 hrs while working a 40hr (or more!) week is a pretty hefty commitment, and I see no reason why that should be solely the burden of the candidate.
You should be able to trust one or two short, simple weed-out conversations (I tend to do one 30 minute "deal-breaker" discussion on non-tech stuff {make sure their and our deal-breakers are met}, then one about 1hr tech screen {fizzbuzz-sorts of things, but more diagnostic}).
After that, you should have a general sense of whether or not they are a) smart and b) can get stuff done. From there, you need to take a risk and bring them in. What is proposed is not fair to the candidate.
We've tried conversations as well, but unfortunately, it is very difficult to objectively measure them into what will make the candidate successful at the company (interested to learn how you do it).
Also, (explicitly) timed coding tests imply that you don't really trust the candidate. Of course they could be lying when you casually ask how much time it took (or whether they got any help), but the few who go that route will probably quickly test out your other filters.
The trust signal, meanwhile, takes a lot of care and consideration to cultivate -- and shouldn't be discounted lightly.
If you see the constraints defined in the post - the goal is to have the interview be remote so that the candidate can do this wherever and whenever they want. And, the tests are specifically designed to be not longer than 4 hours long each. Perhaps what wasn't clear is that this is not in a single session.
Interviews can go for as long as a week - really depends on the candidate's schedule.
"unable to implement a simple linked list"
"Vet the technical competency of full stack rails engineers..."3 hrs of backend/algo testing? rolleyes
4 hrs of frontend? rolleyes
I hope they enjoyed tooting their own horn, yet another place I know to avoid bothering to interview at.
Why didn't the engineers sit around the room and say, "Why do only 1 of 100 candidates that we interview meet our bar? Are we sourcing the right people?" or "Why does it take us 8 hours of someones engineering time to figure out if we want them to meet the team?".
Asking the best candidates–the ones you really want to hire–to do a full day of work just to be able to meet the team is insanity.
Full disclosure: Founder of Mighty Spring (https://www.mightyspring.com) — we're solving exactly these sorts of problems (for both companies and candidates) by turning a truly great recruiting interaction into a web service. Invites are sent out regularly.
TL; DR version:
It isn't fair to throw a company out of a windows because they are serious about their hiring process. It totally depends on their company's growth need! First bullet point is extremely fair but the rest aren't and I agree on that.
Longer version: A lot of hackathon projects become startups and many of them don't require any algorithms. They can use libraries from Python and Ruby to do things they want to do. They can get away with a couple n^3 in their code and customers are okay as long as the service is stable.
And then there are startups out there building products using real algorithms that really require the knowledge of graph and sorting, dynamic programming and etc. If they need an engineer that can build on top of existing algorithms, then that's what they need.
I was a former intern at Mozilla and my interview was very simple and I really appreciate that. But what I do doesn't require me to know any sophisticated algorithm. I can survive with knowing Python and getting around with my brain. But interns on Firefox, servo and Rust probably had to face harder interviews, more tedious algorithmic types (although I heard most of those are basic too, like linked list and talk about C++).
I am also looking for new internship this winter and I fear these algorithmic interviews too, but some companies have serious need and they need people who can at least implement a linked list is important. I don't bother to read what their needs are.
I did roll my eyes when I looked at "3 hrs", "4hrs". That's a lot. But the first item: you don't even know how to write a linked list using your favorite language? Then that's the end. Linked list is not hard - you are not writing in C++ in this case.
The 7.5 hr is a different story, but I won't trash at the first point.
1. How long it has been since you graduated from college. The further away the less likely you will remember how to do it, because you will never do it after college.
or
2. How often you have interviewed for other positions, because that is the exception to 1.
If you were senior, 10+ years why would you be interview at this company? I supposed there are better positions out there for you.
Doesn't make sense with time. I didn't remember how to implement linked list after my class but I picked it up again over the summer as hobby to learn basic algorithm again.
You're saying implementing linked lists are good only if you are trying to hire recent college graduates who will prepare for an interview?
it's like asking someone "Well, explain how list is implemented in Python" when you claim you have been programming with Python years. Well, if you can't even answer that question with a good estimate (it has a double space every time you create a list for example) then you probably shouldn't call yourself a Python guru.
I still remember and will probably never forget how to at least psuedo-code a linked list. But you are right, I have never implemented it after college.
However knowledge of pointers, lists and b-trees have been very helpful multiple times throughout my career.
Indeed. And such glorious moments they are when choosing the right data structure drastically improves performance.
However, if I were interviewing for a Rails developer position, the last thing I'd anticipate having to bone up on is algorithms and data-structures. I'd be expecting to be asked about... well, doing stuff in Ruby on Rails.
Linked list is the simplest data structure one can pick it up in 5 minutes in using languages like Ruby and Python.
Do you honestly think any serious entrepreneurs would want a company growing hire someone who can't even tell you the basic of a linked list (can you imagine one cannot even tell you what a linked list is and the time complexity?)
You didn't even read my comment at all. A lot of things can be done by integrating with the ecosystem out there; but many of the libraries out there are horribly written and there are things you just can't build out of these libraries you find on PyPI.
People who build successful products cannot rely on an ecosystem forever. There are crucial components must be built by engineers using data structures and algorithms we know from textbook and then write custom algorithms that work for the use case.
Too bad. Some people just want to get away with ecosystem all the time; truth is you just cannot.
If a candidate never heard of linked list, wtf does that say about this candidate? Why would I want to hire him if he didn't even attempt to study a few one? I don't expect anyone to write a damn red-black tree algorithm; I can't and I doubt few people is capable of writing that algorithm from scratch (or even remember half of the properties).
Why do we even bother to teach linked list or any algorithm if we can just rely on all the libraries out there?
I decided at the time that I would never do this again, and that's still more or less the case. I might be willing to make a deal - if someone from the company is willing to put a workday into one of my open source projects, even just trying it out and writing up the experience, I might be willing to spend a workday on their programming assignment.
Or, of course, if my family was starving and I was unemployed, I'd do it.
One huge difference here - they do say they'll give detailed feedback. If I had gotten detailed feedback on my programming assignment instead of crickets chirping for a month and then a recruiter call with a one-sentence brush off, that would have taken quite a bit of the edge off. I do give them credit for this.
But still, they are mainly looking for a way to use a candidate's time to their own at an 8:1 ratio. For me, this is a "no apply" condition.
If we assume that only 1 in 100 qualified engineers meets our bar ..
Why not provide the potential candidate with some actual work that you need done?
Personally, I think it's great when companies not only advertise these kind of practices with pride, but then send staff out onto the net to defend them against a veritable tide of criticism. It will certainly help you narrow your pool of applicants, and that is clearly what you are after - it says so in your "how to hire a human" flowchart! Haha.
I'd have to say that's my favorite thing about this process. All too often applicants go away with no idea about what exactly why they weren't a good fit. If someone gets rejected, the least you can do for them is to tell them how, so they can improve themselves.
I've had luck emailing after an unsuccessful interview and saying "Thank you for the opportunity. Informally, is there anything I can work on so that I could be a successful candidate in the future?"
Why would anyone want to work with amazing engineers?
You shouldn't be hiring people because they have experience that matches your tech stack solely because they'll kick ass on the first week.
You should be hiring people that will kick ass 3 weeks in, and a year from now, and 5 years from now -- regardless of whether or not they currently have competence in the nitty gritty details of whatever libwidget you use at this exact moment.
Everyone wants rockstars. But very few companies are worthy.
We don't agree with brainteaser interviews. Having someone build a made up project would be just as bad as a brainteaser. We wouldn't do that to a candidate.
* Worthless Brainteaser: How many golfballs will fit in this Ford?
* Real-world Problem: Let's do cohort analysis on our users and see how our business is doing.
Part of the goal is also a consistent rubric for evaluating candidates.
In the past, we've tried giving every candidate a new problem to solve. It's not fair to them. When each problem is a brand new project it's difficult to evaluate a candidate objectively and consistently.
Really?
The technical portion of the interview process described here is receiving a lot of well-deserved criticism, but I feel this is worth pointing out as well because it's another example of how out-of-touch this company seems to be.
Any experienced candidate knows The Rule (don't drink during the interview process, even at a social event), so an employer that intentionally puts a prospective employee in the position where he or she is effectively asked to do so is incredibly unthoughtful. And foolish to boot, as this could very well become a legal liability.
As if, you know, the candidates don't also have a "bar" that companies have to muster up to. Yet if a candidate where to write a blog post musing that "only 1 in 100 qualified meet my bar", you probably wouldn't want to work with them.
In the end you rather want to hire a brilliant engineer with little experience in the desired technology stack than a mediocre developer that is able to pass your automated tests because he has worked with the technology for years.
You might even get a PR or two. :)
As long as this is made clear in the interview, I see no problem with this process.
Yes. One more non-arrogant company.