Ask HN: How to handle tech interviews?

As far as I can tell, you are far away from @tptacek's post on hiring. He's saying to ignore the resume, have objectively evaluated work-sample tests. You're saying you want people to look at your resume, and that you're really good at talking about stuff. The problem with coding exercise portions of tech interviews (if I'm repeating him correctly) is that they're not as close a representation of actual work as possible, and that they're subjectively evaluated. The standard "tech interview" sits in between your camp and his.

I think we're both on the same page that the standard "tech interview" model is broken. I don't have a problem with doing work sample tests. I have a problem with contrived samples that aren't representative of my coding abilities.

My justification is that I have oodles of experience. I'm happy to prove it -- I just don't think generating an ASCII spiral is a good test of whether I am qualified or not.

It's a screening question. If you make an ASCII spiral that doesn't mean you're qualified, but if you can't do that then you're certainly not.

I cannot off the top of my head tell you how I'd draw an ASCII spiral. But I could tell you how to write a two-phase commit distributed lock, or a C compiler, or an ARM disassembler, or a link-state SPF routing protocol, or, obviously, most typical Rails apps. Am I certainly not qualified, or is that really just a super dumb question?

I don't know where you stand on it, by the way; attribute my hostility to the question, not yourself.

It seems to be a) of no obvious relevance to anything anyone has likely ever done before but b) shouldn't be too hard to sketch out on a piece of paper then turn into functioning code.

First question: can my spiral use only right angles? If so, this gets pretty radically easier.

I also think that if you just draw a grid on the paper in front of you, start at the center of it, and start trying to spiral one cell at a time using ASCII characters you know, you'll quickly find a pattern that can be conveniently coerced into code.

I'd actually code this to demonstrate but I feel like Thomas will be happier if I do code that matters for Starfighter instead.

Indeed, I'd agree with both of you in that I can do a large number of other tasks off the top of my head, but not Tic-Tac-Toe, even though given a fair bit of time to do that "coercing", it's not extremely difficult. However, I'd also agree with kasey_junk's position that this type of problem doesn't quite measure what they think it does, and the "ease" of the problem is really irrelevant. What we should be looking at is the efficacy of the test: does it measure what we think it measures? Does it successfully differentiate between good and bad candidates?

Gonna kinda ramble off here for a second...

I have an intern at my day job now, and we're working through a task I've done a million times. I know how I'd do it, and I could finish the whole thing in about an hour, but I want him to go through all the steps on his own. He's been working on it for over a week, and today he came up with a bunch of possible solutions to a particular obstacle, and he said, "Which one is the right one to do?" And honestly, the answer is that there isn't one right way to go. All of those solutions will work, and at various times in the past, I've tried them all.

What I'm getting at is that I think OP is correct that getting the "right" answer isn't super important.

To take another rather poor analogy, look at Google Maps. If I type in "642 Harrison St, San Francisco, CA", then it drops me at a particular point on the map that just so happens to be an office building. There is no 644 Harrison St, but if I type it into Google, they drop a pin right next to the building, anyway. The interesting thing about this is that Google doesn't care exactly where the building is. Instead, they want to get me close enough to my destination that I can eyeball it and figure out the rest myself.

To put that another way, the "right" answer would be "there's no building here," but Google correctly identifies the problem as "how do I get to this physical spot?" and not "is this a real place?"

Likewise, in a lot of areas of life, not just programming and maps, getting the "right" answer isn't necessarily the only answer. For most things, if you get within a certain range, with a certain margin of error, then that's good enough. If you ask for a medium-rare steak and it comes out medium, it's still gonna taste good, just not perfect. If you run a marathon but wind up walking for 2 minutes in the middle of it, you still covered the distance, even though you technically only ran 98%. If you have a meeting at 3:00 and you arrive at 3:02 or 2:58, it's gonna be fine.

For most things, there's a certain amount of "close enough" that is acceptable.

Does it really matter if 644 Harrison St exists or not? Of course not. Google gets me to that location, and I am able to do the rest from there. Does it really matter if your ASCII spiral is perfect? No, but as long as you demonstrate your mastery of arrays and for-loops and basic algebra, you should pass the test.

At least, that's my two cents.

(Caveat: I think the existing interview process is fundamentally broken, but because asking people to code on a whiteboard is suboptimal not because the specific questions are wrong)

I've asked this specific question in interviews before, many times.

It's like a slightly harder version of FizzBuzz. It appears to have near perfect discrimination between "knows enough programming to use nested for loops with if-statements" and "does not". Programmers (good or bad), once the given the formal problem statement, solve it in under 20 minutes. People who don't actually really know how to program will never finish.

You might be imagining a slightly more difficult question, but it really is easy. It's an arbitrary coding challenge, sure, but one designed to not be tricky on purpose. The naive implementation works great.

I'm willing to guess that question doesn't judge what you think it does.

First, I'd write down the specific things you are trying to determine with this question, and the metrics where by you could judge your employees after you've hired them on those things.

Second, I'd start collecting scores on that question (and all the others you do) and correlating them to "good" employees vs "bad" ones. That won't help you with the false negatives, but its usually enough to point out specifically terrible questions, of which the ASCII circle one is a doozy.

Drop it now, it's hurting your reputation with everyone who you ask it of.

(And if you aren't the originator/user of that question I apologize, substitute "they should" where appropriate)

> How do I convince potential employers to judge me based on my actual experience...

It's a Sellers Market for your services at the moment. You have more power in the interview process than you think.

You might try this word track directly with the Hiring Manager (avoid dealing with low-level HR Flunkies)-- I'm happy to demonstrate my skills and want to respect your time. May I suggest, that we first determine whether or not were a good match for each other. I 'd like to learn more about you and what you're working on. Take a consultative approach, have your own scorecard, and be ready to engage them in a solid business conversation. Think of this as the Discovery Phase.

Sufficiently impressed, the Hiring Manager may have power to forego the silly hoops of the code exercise. But generally speaking, the larger-the-company size the more formalized and mind-numbingly absurd the hiring process. And of course, they want to be 'fair' to the incumbent engineers who submitted to the code exercise when they interviewed.

I tend to try to talk to the Hiring Manager first, but occasionally I get in touch with a recruiter first, and they stubbornly refuse to connect with a hiring manager, and instead go through their rigid process of screening. Any tips on convincing a recruiter to set me up with a phone call to the hiring manager first, rather than to a low-level HR flunky?

> Any tips on convincing a recruiter to set me up with a phone call to the hiring manager first?

Play hard to get...

First, it's helpful to know what level of recruiter you're dealing with, flip the interview on them-- what's their relationship with the Hiring Manager? Are they in-house or a 3rd party firm? Do they have a tech-specific background? How long? What other searches are they working on? Then (assuming the recruiter is a seasoned professional, make a friend); offer you may be able to help them with referrals to your network.

Try this word track-- 'I get calls like yours a lot. The role sounds interesting. But my process to evaluate potential opportunities is to 'always' do a brief phone-screen with the Hiring Manager first. I'm not a prima donna, but I've found this ultimately saves everyone's time and energy. Feel free to share my profile and background notes with him. If you still want to move forward; I can be available for a call Friday morning @ 9:30am"

Take note of the naming convention in the low-level HR flunky's email. Substitute their name with the CTO's name and email them directly outlining your interest in working with them but highlight your concern with their screening process.

>How do I convince potential employers to judge me based on my actual experience and code, rather than contrived coding exercises over Skype?

Consider the person who has the final say in the process. Be it the CTO, CEO, founder, et al. Committing to hiring someone is a big deal. They are taking a risk. Ultimately, it's near impossible to be 100% certain of a persons suitability until they are actually performing in the role itself.

When they are coming to a decision as to whether or not they should hire Candidate X, one of the questions they need answered is "Is this person technically capable". They need something to eliminate that question and all too often they rely on their team asking you to reverse a string or something to that effect.

To answer your actual question, it's important to avoid sounding arrogant but I sincerely suggest you pose your argument as succinctly as you have in your post above and run a mile if you get any objections.

Yes. I don't disagree. Hiring someone is super important, and paying me is very expensive. I don't disagree that they need to be as certain as possible regarding my technical skills. I just need to figure out a way, when asked to reverse a string or what-not, to say "Look, this is Programming 101 stuff, and if you really want to know what I'm capable of, let's look at some projects I've worked on in the past" without coming across as arrogant. (I don't think it's arrogance -- I just think it's not representative of my skill level and experience.)

Address the issue before it comes up. During the initial phone/email conversations where they are suggesting you participate in their interview process, ask them there and then how they assess technical suitability. That's your opportunity to raise your concerns.

Startups in general are less dogmatic in hiring methodologies. We've screened candidates based on very strong personal references, Github, Stack Overflow, etc. However, we've found many people don't have a lot of available work "out there" for us to look at; at which point, we fall back to coffee meetings to establish fit, and technical interviews.

If you want to skip this "hiring manager" game, find startups that are small enough where the people hiring will be the people you'd work with. You'll get a better culture fit idea, and can be a lot more flexible regarding what they expect.

My experience re: startups is similar. However, what if (hypothetically) I don't want to work for a startup? If I want to work for, say, Amazon, in a senior/lead/managerial capacity? Assuming they have a more rigid structure, how do you leap-frog the programming exercise and jump a little further up the chain and demonstrate more advanced skills?

The coding exercises are fine for junior roles, but I think the farther up the seniority ladder you get, the less relevant those tests become. And like many others, I think there's gotta be a better way.

(As an aside, recently I had an interview regarding a tech manager role that was mostly management, less development, and they barely even asked me about technical skills. I wound up passing on the job since it would require moving, but I thought that was interesting. If I apply for a Lead Engineer position in San Francisco, I have to reverse a string or something, but if I apply to be that guy's boss, there's little to no code screening. Very interesting...)

First, let me say, that I view take home coding exercises as a fundamental basis for a good hiring process, as such, I do those with no reservations.

That said, when I encounter one of the oodles of bad technical questions that exist out there, I will frequently ask the interviewer what precisely they are trying to judge and how this particular question judges that. I ask that, because I am unquestionably interested in developer hiring pipelines and I am actively trying to learn about them. Usually, this leads to a discussion that acts as a proxy for what they really want to know, but I make it a point to say how much I do not like these kinds of questions.

The thing is, you can't change someone else's hiring practice. But chances are, if they are engaged in asking questions like that, they have not designed or standardized the pipeline (if they had, it becomes obvious how stupid those questions are). If you are working with a non-standardized, ad hoc process, expressing yourself clearly and professionally is usually enough to get through it (this isn't a good thing).

1. Unless the company specifically reached out to you to fill a critical vacancy, you're likely facing an uphill battle to circumvent the hiring process they've already established. It doesn't hurt to ask for an alternative, but your reluctance will be noted.

2. If you think a company is doing their hiring wrong, do you really want to work for them?

I ask whiteboard coding questions not because I care if the applicant can do some trivial task, but basically to gauge familiarity. Someone who writes code in a language every day for their current job should not have any trouble expressing a simple algorithm on a whiteboard. It's a screening question, and for someone who knows what they're doing, it shouldn't take more than 5-10 minutes. Once that's out of the way, I talk about their body of work, problems they've solved, etc.

The problem is there are a lot of people who didn't do the work but were around for it. They went to all the meetings, they can tell the war stories, they know the jargon, but they just didn't write the actual code.

As for your question about github profile, the problem is that you can't really tell how much work someone did themselves. I'm sure you wouldn't misrepresent your work on github, but people do. People outright lie. They are probably not even being malicious about it, they just don't have a good sense of their own coding ability and feel like if they just post something that they "could have" written.

One applicant who I really liked really struggled writing code on a whiteboard. He described some projects that he had built himself for university, and that sounded really interesting and reasonably complicated. So I took a look through those, hoping that he was just having a bad whiteboard day. But unfortunately it just confirmed, he was smart, creative but just didn't have a lot of practice writing programs. The code was a mess, it did bizarre and roundabout extra steps to do simple things, and from the timestamps, represented months of work for things that should have taken days. I did admire that he was tenacious and didn't give up, and in the end he build something cool but just didn't have enough practice to write production code without supervision. I would have wanted to hire him for an internship, but that wasn't was we were looking for.

> I ask whiteboard coding questions not because I care if the applicant can do some trivial task, but basically to gauge familiarity. Someone who writes code in a language every day for their current job should not have any trouble expressing a simple algorithm on a whiteboard. It's a screening question, and for someone who knows what they're doing, it shouldn't take more than 5-10 minutes.

It's not that simple for people who really can't interview. I am one.

Last week, I did an interview with Triplebyte, and I got to pick essentially an AI for Tic-tac-toe game. I know that I can brute force the game state with a recursion. I know how to write a recursive function: an exit condition, a loop for recursive along with rollback and retry. The problem is when I'm actually writing the code, my mind blank out on any and all details: I know there is an "if" there, what it should check out, but I can't put the actual condition in my mind (is it "x == y" or "x!=y" or "x==1"). So I'm screwed

I know I can write the actual code, because I was able to do that right after the interview is done.

The ASCII spiral one was from Triplebyte as well.

The problem with the task is that you can demonstrate decrementing counters, recursive functions, and familiarity with the language, but if you don't solve the specified problem in a constrained timeframe, you fail. (My feedback was basically "You clearly know how to code, but you didn't solve the problem.")

One of the problems with the Tic-Tac-Toe AI scenario is that you have to figure out an algorithm to win at Tic-Tac-Toe. (If I recall correctly, one of the requirements is that it beats you.) I know how to play -- but to win every time? I'd have to formalize an algorithm, then convert it into working code, within a strict time limit with someone watching over my shoulder. I can do it, but probably not in 30 minutes. It'd take me 30 minutes to set up my 2D array and accept input to populate the board.

That's not how my day job works. That's not how any programming job works.

When I get a problem at my day job, I know the entire domain. I know who my customers are, I know the problems they have. If my day job were Tic-Tac-Toe, I'd know everything there is to know about how to play and win, and then it's a formality to convert that into code.

If you really want to go with the coding exercise thing, you can't focus on the "right answer." Whether their tic-tac-toe program wins every time is irrelevant. What's relevant is that they jump straight to the notion of a 2D array, that they know how to do recursion or for-loops. What's relevant is that they can write code without too many syntax errors, that they're comfortable, and that they make progress throughout the time alotted. Whether they actually cross the finish line is irrelevant at best. It's a red herring.

In the end, do you want to know if I can code, or do you want to know if I can write a winning Tic-Tac-Toe algorithm in 30 minutes? Those are two different answers -- just because I can't do the latter doesn't mean I'm not qualified. I just means I didn't pass your test.

Hey, another Triplebyte founder here. This is something we've talked a lot about. To set a balance, we talk to people about a project that they've worked on in the past, and also have them pick a problem from a list, and work on it with us over screenshare. I'm sorry this did not go well for you (I'm curious to look at our notes from your interview, but I don't know who you are).

We're focusing on being consistent and fair. I understand (and am sorry) that we mess up and miss good people. But looking exclusively at experience creates other opportunities for errors (I believe more opportunities). We don't require perfection on any of the problems. We're looking for process, and try to help people (for example, on both of the problems mentioned, we generally talk about good approaches before the person starts coding). But again, we definitely mess up. Sorry about that. We're trying to do it less as we improve the process.

How is it fair to ask somebody to program tic tac toe? I promise you you'd shit your pants it you tried to do the programming I am currently doing (matrices with 1/2 million element diagonals, optimal estimation, and more linear algebra goodness), yet I haven't got a clue how to start a tic tac toe problem.

Now I realise it must be something fairly trivially obvious, but it is not coming to mind at the moment. So, I'm screwed. I don't write tic tac toe games at work, I don't write them for fun, and I don't write that sort of thing for the most part. I'd probably try for a backtracking algorithm, but that is easy enough to mess up in an interview situation.

Throw in a 'helpful' comment or two if I seem to be off track and now I'm sitting there trying to figure out if my idea is wrong, if it is right but the interviewer is not looking for that approach or perhaps doesn't understand it, and so on. It just sounds miserable. Of course, if I had just done another interview with tic tac toe, failed, then googled it, I could probably just write it letter perfect for you, but I'd act like I was puzzling it out to impress you. What have you learned from that charade? Or another person just graduated from a boot camp where they programmed nim, towers of hanoi, and other stuff, and quickly scratch out a half working example. That person is going to do better than somebody that can read research papers, implement them, prove the correctness of their implementation, work with hardware, design circuit boards, develop a robust software suite that is testable, extensible, and fast, delivers on time, and doesn't waste time over-engineering things. Who do you want on your team? And who is 'tic tac toe' going to select for?

Ah, and I googled it. Perhaps it isn't that intrinsically hard, you don't need a decision tree or anything, but here also is a 28 page (28 page!) chapter from a book from a Professor at Berkeley on implementing tic tac toe.

https://www.cs.berkeley.edu/~bh/pdf/v1ch06.pdf

Well now I can pass your interview. Did I become a different person in 30 minutes?

Just for your notes, I did not get a "good approaches" discussion. We just jumped straight into it. So much for consistency?

I don't really want to go any further in public. I already sent you my feedback (and received no response, for whatever that's worth). Good luck in your future endeavors, and thanks for your reply.

Thanks for interviewing with us. We know interviews are stressful and we try and account for that by offering a selection of different problems. Also, if we think someone is freezing from nerves, we’ll offer them a problem to do on their own time after the interview before making a decision. We know we can continue doing more and want to run more experiments around this for our next batch of applicants.

What's interesting to us in this specific case is that you actually did great during the interview. We've done hundreds of them in the past few weeks and we're really excited to talk with you again for the second round.

Just to be clear, I think that interviewing with Triplebyte was the best experience in interview I've done so far, and I would definitely recommend Triplebyte impromptu for anyone looking for job. I vow to never interview with Google again, and I'd definitely try again with you guys if I bombed the interview.

The questions, despite certain drawbacks, are quite reasonable in term of scope and requirement. And I guess the biggest difference is in the interviewer attitude: "finding strengtha rather than weaknesses" and actually care about the interview itself. I don't know if it will translate into successful hire, but certainly leave a much better impression on the candiates.

You don't. Pick an employer on the same wavelength as you.

JFWIW: I'm @tqbf on Twitter, and 'tptacek here. @tptacek is someone else. :)

(About me: I was an early engineer at one of the fastest growing SaaS company on the planet. I had the privilege of contributing to our interview process that took us from 5 to 250 engineers. I'm now trying it from the candidate side: http://InterviewKickstart.com)

tl;dr

You have three choices:

1. Apply with a strong referral and/or

2. Apply to exactly the same kind of jobs/projects that you have had work experience in, and/or

3. Apply to a company that's hiring only one or two people a quarter (slow, careful)

A bit more

Everyone hates the tech interview process. Everyone. Including people on the other side.

Let's understand the process, by taking the most typical case, where a candidate with experience in X type of projects, is applying for a job with Y type of projects.

e.g. If you have worked on say Payment Systems' projects for last 5 years and you are now applying for Deployment Systems' role, then as an interviewer, how do I evaluate you? I don't know enough about Payment Systems to judge your work of 5 years in 45 minutes. And I can't presume you know enough about Deployment Systems that I can ask you questions built on my knowledge of several years.

I hence have the following options:

1. I ask someone about you. That's possible, but not systematic, because I need to find a person that I trust very well. It's much better if that person refers you. Hence your option #1.

2. I can hope that you and I have some project in common in the same domain. That's getting much harder these days, as programming is getting very vast. Hence your second option. Then I can evaluate your work and decide one way or another, quickly.

3. I can look at your work of a different domain and give it a proper evaluation. That's takes time, often hours and days. I can't spend that much time, when my goal is to hire 20 engineers a quarter (and hence interview 5x more). Not to mention that evaluation is also subjective, error-prone and contentious. Everyone has different ideas on design and code-quality. Add that to the fact that I need your work evaluated by everyone in the team, not just me. It just becomes untenable. Hence your third option (apply to a place that's hiring slowly).

Being human, hence, I have to fall back to the path of least resistance. That path is to find the lowest common denominator of our knowledge, which can be evaluated in a short amount of time. Yes, it's not perfect and it's still a bit subjective, but there aren't that many ways of writing quicksort <or insert your favorite question>, you know.

I hence ask you that question, begrudgingly, knowing fully well, that I'm myself going to be subjected to the same treatment when I go out interview. But at least now I know why, and I look for the best I can in the current process.

Humans.

Interesting. Someone downvoted this comment. Not sure why. IIUC, this answers OP's question and also gives an explanation. Downvoter care to explain?

It's frustrating, isn't it?

I've said a few times here on HN that it's a career goal of mine to get to the point where I don't have to do technical interviews. I'm not particularly offended by tech tests[1], I just wanted to get to the point in my career where I didn't have to take them anymore. Unfortunately, I'm not sure this is realistic anymore, at least not without severely limiting my opportunities.

Overall, I'd say that if you really don't want to take another tech interview test, your options are:

1) Consult to hire. This is unappealing for a number of reasons, and many companies will still ask you to take a technical test to work as tech consultant anyway. But they may be more willing to take a chance on you as a consultant without the tech test[1], so you might be able to prove your technical ability on the job and get an offer that way. I'd say this is an option if you are unemployed, failing at the tech test part of the interview, and would like to try something new.

2) Work on open source projects that span multiple institutions, and get hired by one of those institutions. This is different from having a generally impressive github profile or history of open source contributions - it is more about having worked directly on an open source project with developers or other staff specifically at the company or institution where you are interviewing. Why ask you to find a cycle in a linked list if you've already developed and tested features directly with the devs at the institution where are interviewing? This is similar to the "consult to hire" - there are people who have worked directly with you already and know you can do the job.

3) Become a luminary. This isn't an option for me, but the people who created Python, Ruby, Rails, that sort of thing, they can don't have to find the longest path of non-negative integers in a binary tree at the whiteboard to get a job. Of course these folks don't exactly need advice from me, and I doubt they "get a job" in any sense that resembles the way I "get a job".

I'd say that for your good but not famous dev, the second option can be effective, though you do need to make an effort to keep branching out and looking for the right kind of project (they aren't necessarily easy to come by). Personally, I've reached the conclusion that I'm limiting my options too much by avoiding technical interview exams. Studying those data structures and algorithms books, getting razor sharp with solving medium to difficult problems at a whiteboard in 45 minutes, it's probably just part of the profession, at least for now. You really do lose a lot by refusing to participate.

[1] by "tech test", I mean the sort of questions you'd find in "cracking the coding interview" or questions you'd typical find in an algorithms and data structures exam (variants on trees, hashes, graphs, sorting, run time), along with some OS stuff involving threads, deadlock, and so forth).