Startup Interviewing is Fucked | Hacker News Reader

Startup Interviewing is Fucked

zachholman.com

99 pointsjuokaz27 comments

Open articleView on HN

It's a very difficult situation. I know we spend a lot of time discussing this on HN, but I'm glad we do, this really is one of the biggest issues facing our industry. We (software developers) essentially have to study for and pass a comprehensive whiteboard exam almost every time we apply for new job. Alternatively, we may have to spend up to a day or more on a "take-home" exam.

I do think that Gayle Laakman McDowell (who wrote "Cracking the Coding Interview") has written a couple of good posts on this topic.

http://www.gayle.com/blog/2015/6/10/developer-interviews-are...

It's a pretty good roundup of the various interviewing approaches, and the problems with each. I have some differences of opinion, but I agree with the overall point, which is that there really isn't a great way to do this. The in-person coding exam (I don't like to call it an "interview" because I really do think it is more reasonable to call it an exam) at the whiteboard may not be the worst of a bunch of bad choices.

Here's my main problem with all of this - it is extremely unpleasant, developers clearly don't like it, employers acknowledge that it has a very high false negative rate… yet seem to see no connection between this and their own constant complaints about a "shortage" of software developers.

I do see my friends in other industries interview, and they don't go through anything like this. Lawyers, doctors, nurses, and actuaries (among others) certain do take exams, but they take those exams under much more controlled circumstances, where well understood degree programs and study paths prepare you for the exams you will be taking (even so, many people describe these exams as the most stressful academic experiences they've gone through).

All in all, I do think that this practice deters people from becoming software developers, staying in the field, or changing jobs. It's one of those things that makes individual sense for a single employer, but in aggregate takes a heavy toll on the field.

I do think some kind of highly respected professional accreditation could solve this problem (actuaries, for instance, must pass a rigorous exam on vector calculus, linear algebra, differential equations, and so forth), but I'm guessing that senior actuaries don't have to do a tricky integration by parts during an interview. Devs most definitely do. Then again, I fully recognize that "professionalization" does come with serious risks (with a few edge case exceptions, anyone who wants to become a lawyer has to go through three years of law school at $100,000+ cost, and many many people feel this is excessive and is really there for regulatory capture/cartel building).

Again, there's no perfect answer. My only thing I can say with real conviction is that companies that engage in this kind of interview exam, accepting that it is stressful and produces a high false negative rate, really need to stop scratching their heads about the "shortage" of good candidates. I find that a little hard to take. I accept that there are good reasons to do these exams, but you must understand the role you have played in your own shortage when, by your own admission, your process weeds out lots of good candidates.

And one other thing - whatever the process, employers should not look for ways to push the inefficiencies of the interview process out onto the candidate. Although I just didn't study enough for my google interview exam (and who knows, I may not pass even if I took 6 months to study), Google clearly spent the valuable time of its own engineers to interview me. I've done a take-home exam where I spent 8+ hours, and honestly, I received so little feedback I have no idea if they even read it, it was a one-line brush off a full month later ("we've decided not to proceed with your application at this time.").

Another good post from Gayle McDowell on this one:

http://www.gayle.com/blog/2013/09/18/companies-who-give-cand...

That I won't do anymore. It has cost me some opportunities, but I do sort of feel we need to make some kind of stand as a "profession" (it is very difficult to make this stand as a scattered collection of individuals, which is why I put "profession" in quotes).

Thanks for the link to Gayle's post. This is what I have concluded also. Near the beginning of our hiring process I explain to the candidate all the steps in the process and why each step exists. I acknowledge the process isn't perfect and express gratitude for the candidate bearing with our imperfect process. I also ask the candidate if she has any questions or concerns before we move on to the next step. The vast majority of feedback on the process I receive from candidates is positive, and good candidates do persevere to the end.

I'm torn on homework assignments. On one hand, Gayle's point is a good one: they're asking for a big time commitment from candidates. On the other hand, they seem to be the challenge that maps most directly onto actual work performance.

Our process here is:

1. Non-tech phone screen 2. Tech phone screen 3. Homework 4. Onsite

The onsite is, to many candidates, surprisingly light. This is because the homework makes up most of how we're evaluating your tech skills. The onsite is just to make sure that you wrote the code you submitted, and then to test your non-coding skills (architecture, communication, working with non-engineers, etc).

But we do lose some number of candidates who just don't get around to doing the homework (or possibly, as you do, refuse to do it). This worries me, since I suspect the people least willing to do homework will be the most senior and/or those with the most competing offers.

We've recently started experimenting with allowing candidates to chose between homework and a code sample. The idea is that a sufficiently complex and interesting coding sample should tell us about as much as coding homework. It involves more time on our part to evaluate it, but less for the candidate to prepare it. The early results have been mixed (a lot of unsuitable coding samples), but we're iterating on how we ask for coding samples to try to get better quality ones.

The last time I've done homework I gave the candidate a choice of problems. As in "pick one of these, solve the problem, and explain why you chose it".

Some were larger than others, some were very simple and straightforward while others were more amorphous.

As an also-Director of Engineering applicant, I'm at the point where if I get to the onsite and you ask me beginner Javascript questions or give me riddles, I'm inclined to say thanks but no thanks and move on. Not literally, but it's definitely not going to make a great impression on me. After all, part of the reason you're hiring a Director of Engineering is to... well, direct engineering, including the interview process. Still, it's a red flag.

I would also chime in to agree with @trek. I've followed the careers of people my teams have passed on, and they're almost all wildly successful. I've been happy with 75% of my hires. I obviously haven't figured it all out, and my teams have definitely let some good ones slip through.

Surely we can do better.

It reminds me of an observation I read the other day. Paraphrasing: "When I am an applicant, I think I'm being interviewed by geniuses. When I'm the interviewer, I'm ecstatic if they can write 10 lines of code."

Ain't that the truth.

Yep - most startups use hiring as a cargo cult ritual that allows them to avoid the hard (and scary) problem of figuring out how to make a product people need or want. With hiring, you get to imagine you're in control (you're not)

Whoh! Did I just read that Zach Holman needed technical interview to get hired? In that case, yes, things are seriously broken.

His idea of pair programming as interview is actually pretty good. However I would stress that there is no such thing as bad interviews, just bad interviewers. I've seen cases of "one secret right answer", "question with hidden land mines" and "irrelevant puzzles that you would never need to solve on actual job". My favorite strategy is to think of some problem I solved myself as part of the job, abstract it out a bit and see how candidate would go at it. This makes things very relevant, allows me to compare candidate's thinking and performance with my own and avoids asking popular chewed up practice puzzles. The problem is that this puts a burden on interviewer to first think of great problem they had solved and then nicely make it simple, interesting and friendly question. This is much harder than looking up one of the irrelevant popular puzzles asking one of them.

Zach's lucky (privileged, even) to have a platform to rant on and a large number of connections to find his next gig. He's unlikely to lose any opportunities as a result of this post.

What about the rest of us, who don't have "star-power" or anywhere near the kind of platform that Zach has? If you're just Joe (or Jane)-average programmer, yes, startup interviewing is fucked. Doubly so if you don't live in SF or SV.

I agree with you, but I have heard the opposite from some people. They like being able to memorize programming interview books and grind problems on LeetCode/HackerRank instead of having a portfolio or other alternatives.

I may have to break down and hop on HackerRank. Have you heard of https://codility.com/?

I haven't but I will check it out!

From my time at /r/cscareerquestions, LeetCode is the "hardest" and thus the best for top companies. HackerRank is good but every challenge requires parsing STDIN instead of just giving arguments to a function the user writes. It's very annoying to me and a deal breaker, especially with certain languages.

yeah the first time I had an technical interview with hackerrank I had to go google how to do i/o in Python

I can't understand why HackerRank gained such popularity. Among many similar sites it has lowest quality of problems, especially input/output specification. The "pay for more test suites" strongly reminds me Pay to Win ...

If you don't know how to solve a simple problem, especially one that was solved 50 years ago in computer science, you're fucked anyway.

A lot of these interview questions are 'crap' but designed to be solvable in 20-30 minutes and not require hours of domain specific knowledge. In other words, it's something _anyone_ with _any_ degree of problem solving skills should be able to solve.

The problem isn't that interview questions are too simple. Sure, quick weed out questions that can easily be sent out before an onsite interview make sense. But to have someone implement a red black tree, or implement some other algorithm that has an "elegant recursive solution" that is so far removed from any actual work the programmer would be doing is asinine. It's the same reason standardized testing is such a poor measure for many kinds of intelligence. It's forced and impractical- far removed from the real world.

Onsite interviews should consist of working with another engineer to look at a real problem the company is dealing with (or was dealing with in the past) to see how the interviewee approaches the problem and tries to solve it. It's not about reaching a solution per se, but rather how well the candidate works with the team and if her approach is the correct one.

> Zach's lucky (privileged, even)

Eh, he built that platform. Not sure I would call it luck or privilege.

I am a huge fan of the pair-programming interview, as it gives the candidate a clear view of what they're getting into and gives you a chance to hear how their brain works. If you feel it's exploitative, there's nothing to stop you from paying them for their time. They might not take the job, but they will probably tell their friends about the place with the wacked-out interview process.

Speaking of non-real problems, one interviewer asked me to draw a diamond with asterisks.

This is apparently a pretty common question. I've always wondered where it would apply in the real world in my field, maybe someone can help.

It's more of an ego/race/etc issue with some. I've had instances where luckily I read, typed & memorized the interview algorithm the night before, which I regurgitated on the white board. Only to to be "pfffft that will never work" by the interviewer.

It's a variation of Pascal's triangle.

It's probably a common question because it's a common CompSci homework problem.

Hiring is hard. You're trying to meet someone, and evaluate them in a short period of time. I get it. Multiply that by many candidates and...

I care way more about personality than technical skill. Technical chops need to be adequate for sure, but anything above that is just a bonus. If you hire the right personality, adequate tech chops is just a temporary problem.

One of the best articles explaining why ENGINEER interviewing is fucked is summed up in this life-changing article. I say life-changing because after you read it, it will change the way you think about engineering jobs at companies with managers.

How the Other Half Works: an Adventure in the Low Status of Software Engineers https://web.archive.org/web/20150919055214/https://michaeloc...

Zach Holman would be particularly interested in that article since it was written by someone who promotes the idea of Open Allocation (i.e. companies without managers)—which is how GitHub used to work when Holman was there.

It's only when you see the bullshit side of managerial status that it becomes apparent why Engineering interviewing is fucked.

I actually got bit by the opposite of this early in my career - I have a knack for solving algorithmic coding challenges, which meant interviews tended to go really well, and led me to think my self-taught web dev skills were much better than they were.

As Zach mentioned, quite a few interviewers like to use recursion-based puzzlers, and I actually enjoyed those - I seem to have one of those brains which is wired up for understanding recursion. On the other hand, I'm actually more drawn towards the product/UI side of things than the hardcore algorithmic tech innovation. These days I rely much more on discipline than on natural talent.

Personally, the only bad interview experiences I've had were not at startups, but at the big name co's. There's maybe one exception to this, a startup (extremely well funded by bay area standards) that was arrogant enough to expect me to show up without confirming my availability. They hired someone else who did show up on that day and didn't bother to tell me about it. I wouldn't want to work in such a company anyway.

Mostly though, startups have made it a really amazing experience. They have made a real effort to fit the misfits and I owe my career to them.

I have notions about who deserves the job.

And it's probably someone not too dissimilar from myself. What a coincidence!

From 10,000 ft we probably hire like the McDonalds down the street. Or that law firm downtown.

https://lab.getbase.com/cracking-java-base-coding-interview/ Base's view on interviewing ;)

A startup founder recently asked me to do a 45 minute problem solving Facebook/Google style interview over the phone. If I wanted to do a Facebook/Google interview I would go interview at Facebook or Google.

I don't think it's just start ups all technology hiring is all about problem solving when 90% of it is about spewing code. which to me is probably why there is so much churn. In consulting there's an old saying about managing expectations. I think hiring geniuses to spew out code is poorly managing expectations.

Reply on news.ycombinator.com