Hacker News Reader

Comment by fecak | Hacker News Reader

"If hiring is broken, what else is broken?"

I've seen quite a few articles on HN recently where what we call "candidate experience" during the hiring process was a theme. Whether this writer chose to list "Recruiting" as the first topic because of chronology (recruiting was first experience with company) or to emphasize the importance is unknown, but candidates today seem to assume that a broken hiring process is indicative of flaws in culture and how the company values employees.

Companies that simply play the numbers game and don't consider improving candidate experience are running the risk of reducing their brand as an employer, regardless of how the public regards their products or services.

View on HN

It's important for candidates not to over-emphasize the smoothness of the hiring process.

I almost didn't work for a company because my experience during an internship was that HR, payroll, and other departments that were not part of my day-to-day job were painful and bureaucratic. I was talked into the job anyway, and it was without a doubt the right decision for me. I learned an enormous amount, worked with some terrific people, and helped ship a very successful product. You could argue that the ancillary stuff reflected company issues that ultimately led to the company's demise. But I would certainly not have passed up the 3 years I spent there even if I'd known that in 3 years I'd end up looking for a new job. And that's not least because so everyone I worked with did the same thing, and now I'm still working with many of those same people on much of the same technologies.

What was important about the candidate experience? It was clear that the team was interested in working with me and developing me as an engineer, and that they were very bright and working on important problems. That trumped basically everything for me.

Because we spend so much time at work and because a bad workplace is such a miserable experience, I think candidates should pay close attention to negative signals, even if they could turn out to be anomalous. Of course, when things go wrong it's certainly important to pay close attention to how that is handled. Basically everything that happens during the interview and recruitment process should be treated as signal even though some of it will be noise.

For employers evaluating a candidate, "if it's not yes it's no" - I would suggest job seekers to have the same attitude. (Unless your current job already sucks.)

It's chronological, although I did want to emphasize this issue in recruitment.

First impressions count.

It's hard to know what a company is really like without knowing some insiders. Searching on the Internet may find opinions pointing every which way. And then, you meet their recruitment team, and get your first -- and maybe last -- firsthand exposure to the internal workings of the company.

Hiring the right staff is obviously very important. So any issues with the hiring process that cannot be fixed is deeply worrying. Does the company not value hiring? Why can't it be fixed? Who will my co-workers be? What else is broken? etc.

I regret that I didn't walk out during the misaligned interviews, to save everyone's time. I did eventually tell recruiters not to contact me anymore, but that seemed to make them more interested, and I was handed to a different team. (In the end, I picked Netflix anyway, and am very happy I did.)

Netflix recruiting was awesome, and I think other companies can do the same. Don't put up with recruiting problems -- help fix it. It may be your company's first and last impression.

I've dealt with many companies large and small (retained recruitment, tech) that have admitted early on to flaws in their hiring process. Typically, I find it's someone in engineering all the way up to the CTO that admits the process is broken, and they generally want to circumvent internal HR processes to get the candidate to their potential boss and co-workers as efficiently as possible.

HR typically wants to control as much of the process as possible, which often means an HR screening first. This tends to turn off engineers considerably, as it's deemed a waste of time, so the first step in the process is perceived negatively and that negative first impression is hard to overcome. The clients I've represented that have the smoothest process are those that use HR to facilitate and provide answers to questions, but to try and limit involvement beyond that.

Do Netflix's interviews have any trick questions (e.g. manhole covers) or coding questions that have nothing to do with the job (e.g. Write a function that can detect a cycle in a linked list)?

I would never interview at major tech' companies because from what I've read of their interview process, it is all about knowing random Computer Science trivia and studying a book of trick questions.

I like more traditional corporations/business who give you questions actually applicable to the job and your experience. Silicon Valley interviews are pretty insufferable, due to their pseudo-intellectual elitism.

Companies should take their interview, and then give it to what they consider their most successful employees. If the employees wouldn't be "hired" then the interview process is completely broken.

As always, it depends on the group or team you are applying for. Some groups actually have a lot of applied computer science concepts and benefit from these type of questions. On my team, Web UI, we typically try to ask questions that pertain to the tasks they would encounter day to day.

I feel your pain on coding questions which have nothing to do with the job, but this can be difficult.

We recently tried to identify such a question, with a scope such that the candidate should take about an hour and a half to complete - where complete also implies documentation, tests, etc. We couldn't find an appropriate question which would be something that would be highly indicative of the specific work they'd be doing that a) Didn't require a ton of domain knowledge (we don't expect people to have that domain knowledge coming in) or b) Would be too specific to particular technologies such as which ORM type library one is using, which type of DB, what libraries, etc.

Typically when I've seen examples of these questions where they are fully relevant they're in a space that's tightly defined - you're using say RoR, MySql, etc and candidates are expected to be proficient in those. That's not us.

I'm a hiring manager at Netflix and spend a bunch of time talking to other hiring managers at Netflix.

I don't know anyone who does trick questions (most people I know find them anathema, and even Google -- famous for them -- acknowledges they're useless). I definitely do know some people ask the 'cycle in a linked list' question (or others like it); honestly, I don't see a problem with that set of questions.

even Google -- famous for them -- acknowledges they're useless

When I interviewed at Google (four years ago now I think) there were no trick questions. Hard questions? Yes. But none that had "gotcha" answers that you'd either have to know already or be some kind of genius to come up with on the spot in an interview.

Can I ask for your thoughts on the cycle in a linked list question?

I asked someone that today, and it didn't go very well. I gave him the hint to start with writing code to print out all of the children of a node, then when that stack overflowed, he was able to make progress.

Do you think there are senior generalists engineers who would be great who could struggle with that question? Its hard for me to put myself in those shoes but I believe it could be possible.

The goal of that question from my perspective is to watch them solve a problem that requires recursion. I've always found that people who do well on that stuff tend to get stuck less designing software in general.

I do worry that it cuts out people who would actually be good.

I'm very interested to hear thoughts on this from others.

It depends on how you ask it. If you ask it as an exploratory question, it can be useful. If you ask it as a question where you're expecting the "right answer", it's awful for interviewing, because the insight is difficult to spontaneously come up with, yet easy to have already encountered online. Neither result is very informative.

It seems to me to still be very poorly understood in general that the purpose of a question in an interview is not to see if the candidate gets the right answer, but to see how they function. (In those interested enough to read the discussion this deeply it's probably somewhat better understood.)

The reason I ask the question is that I want to see if the person can handle thinking about recursion, and how their debugging process works, and how they built toward a solution.

I always stipulate that they can use whatever brute force approach they want, and that i'm not judging on the algorithm. (Though we'll then discuss the big O complexity of what they've done and i'll ask them to brainstorm some avenues for optimization).

The main drawback of this particular problem is that it's biased toward people with more formal CS experience who have probably seen more tree and graph stuff, when i'm not really trying to measure for that experience directly.

I'd be interested in a problem that doesn't have that aspect but still takes up a fair bit of brain space when thinking through. What I mean by that is that when you're trying to build and debug something recursive, you have to hold in your mind some representation of what the stack will be at various points, and not everyone has a great aptitude for that.

For what it's worth, the solution to that question was an open question in computer science for over ten years, so I would guess that people who get it quickly in an interview are more likely to have heard the answer before than successfully invent the tortoise-and-hare algorithm on the spot. Further reading on that question in this old thread: https://news.ycombinator.com/item?id=7953725.

In practice, what happens is that we both acknowledge that if this was a real work problem, step 1 is to google it. I then tell the candidate that i'm not going to judge them on the algorithm they use, just their ability to work through the problem. This isn't a whiteboard problem, it's done on a machine.

It's never happened to me yet with this type of problem, but i'd be really impressed if I did run into someone who has the optimal algorithm for this memorized and can code it on demand in an interview. I'd certainly want to check that they haven't spent all of their time just memorizing algorithms and they can still do other things of course.

If this did happen i'd want to validate that they really understand the solution and aren't just spamming it out by rote, but if they've really got a few algorithms in the bank like this i'll just consider this box checked and move onto other considerations.

all I really want to know is if the candidate has the "horsepower" to deal with these sorts of problems, many people don't. someone who can code optimal solutions to these things on demand obviously does.

That said, obviously this is much easier for someone who's dealt with trees and graphs and such a lot before, so it's not ideal. I'd certainly be interested in a better way.

Would you accept with full credit an answer like this?

Iterate through the list adding its nodes to a set. If a node is already present when we go to add it, then stop, since we've found a cycle. Otherwise if we reach the end there is no cycle. With a hash set which has O(1) insert and membership check, the algorithm takes O(n) time and space since.

Note that I mean we're adding the nodes themselves to the set, not the element stored by / pointed to by the nodes. This may mean storing a pointer to the node in the set, or some other kind of object identifier.

I think this solution is fair to expect from anyone, including the caveat about node identity. Plus a candidate who already knows tortoise and hare should be able to come up with this when asked to solve the problem another way ("what's the simplest, most naive solution you can think of?"). I am not sure that I would expect anyone to come up with solutions beyond this one and the O(n^2) solution during an interview. I think those solutions correspond to knowledge that everyone should have in their working set. "Have I seen this thing before" is a pretty simple application of data structures. The O(n^2) solution trades off storage to track whether a node has been seen with determining it on demand in duplicate each time by traversing the list from the beginning repeatedly.

(If solutions are fair game that involve modifying the list, such as reversing it, or assuming that list nodes can be marked as seen, then make sure the boundary of what's allowed is clearly communicated. Make sure that the candidate does not neglect to consider a category of solutions by mistaking it as out of bounds. Candidates who are asked to design an algorithm to process a data structure cannot usually modify that structure to make the problem easier, so if this is something that you want to encourage, then you might tip the candidate in that direction by asking them to write the definition of their list node, and encouraging them to implement the node however they want as long as it meets your definition of being a linked list. Conversely if this is out of bounds, it might be best to provide the definition of a list node.)

Anyway, it seems like a fair simple interview question along the lines of a medium difficulty FizzBuzz, if the answers above give full credit. I don't think effective software engineers would be tripped up by this. One area of concern might be people looking for overly fancy solutions without trying "er... can I just track what nodes I've seen in a set?", but finding simple solutions is part of being a good engineer. I think there are better questions than this one: (1) it's too easy to memorize all the answers (2) it's already well known by this point (3) I have not heard of a good way to build on the question to make it challenging and interesting for high performing candidates. This might be an OK intro question before moving onto a harder one. I've never asked this in an interview - just going by instinct.

Yes, someone who can code the solution you describe on demand would be a flying colors pass for me.

I agree with you that there are probably better questions. As you say, I'm looking for a medium difficulty fizzbuzz. I also agree with you that this is not going to be challenging and interesting for high performing candidates.

Incidentally, if someone has managed to memorize the solution to this and other algorithm problems I would consider that a pass. just the ability to do that is enough. there's enough code involved that they have to understand the problem pretty well to memorize it, and its the ability to understand that i'm interested in. (Barring some extreme outlier with super memory but bad other skills, but I'll have to detect that with other questions).

The problem I see with your question is that you have a bias towards a solution.

"The goal of that question from my perspective is to watch them solve a problem that requires recursion"

I wouldn't use recursion for my solution. It takes up stack space for no reason. If I'm going to be allowed to use more space, I'd just loop through the list, add them to a set, and check if I've already seen it. No recursion, but extra space. Is it more space than another stack frame? Depends on the type of set I'm using. But at least I know I won't blow up the stack (maybe the heap :-)

Since you are asking for opinions from others, I'm going to assume you wouldn't be upset with me solving it this way. So am I less of an engineer because I didn't immediately go for recursion?

With that said, everyone has a baseline - answers they expect to see. But everyone has to keep an open mind.

My company is trying to come up with a corpus of questions and answers to those questions at the junior, normal, and senior level. I've voiced my opinion that it is a slippery slope. If you get someone who isn't a skilled interviewer, and the interviewee doesn't use one of the prescribed 3 answers, what happens to that candidate? Granted it is one data point in the loop, and not the only data point. But as I stated, it becomes easier to be "lazy" during the interview.

>So am I less of an engineer because I didn't immediately go >for recursion?

No, not at all. a stack is a stack, whether it's the call stack or one you maintain in code.

Its not the recursion that's important to me, it's holding a problem of moderate complexity in your head in order to be able to come up with a solution. Problems that lend themselves to recursion seem to present a bar that some engineers struggle with, and others find easier, in my anecdotal experience.

I know I said recursion is important all over the place, clearly my thinking is sloppy on this and needs to improve.

I have this idea that there is a class of problem, that lends itself to recursion, that separates the wheat from the chaff in some ways. maybe that's nonsense.

What I really want to test for is the ability to juggle abstractions while working through code problems. there's probably a better way to do it.

Thinking about this corpus of questions and answers, I guess the common idea of fixing broken code might have a bit more merit because it removes the risk of an interview focusing too much on judging the approach and turning it into a trivia contest (have you seen this before and do you remember how to do it).

You are trying to measure what?

I'm going to assume your answer is "on the job performance". If not, this is going to be a difficult conversation, and I'd ask why you are trying to measure for something you aren't going to be paying for.

The best predictor of on the job performance is ... on the job performance. Everyone but the most junior person has experience. Look at that.

All of these exam type questions usually rely on knowing a trick, or being recently well versed. I TA'd a graduate course in Algorithms. That's rather relevant, I would think, since you aren't offered that if you don't have ability. Yet you know the last time I've done, say, dynamic programming? It's been awhile. I know about it, I know when to decide whether to pull it out, but ask me to do it, in a high pressure interview (all interviews are high pressure), and well, I'll probably fail. I'm working on other things like Bayesian inference. Bet you'd fail doing that stuff, even though it is just simple multiplications and such, when you get down to the implementation. Unless, of course, you are working on that sort of stuff right now.

I find the situation in SV somewhat ridiculous. To go out and interview means a lot of prep. Google recommends spending a month reviewing algorithms before applying. You know, I have an actual job, and I have open source side projects, and I have a family, and hobbies, and an old dog I want to spend all my time with. It is trivial to find out my skills by looking at what I have done and asking me questions about it. Have me give you a 1-2 hour tech talk, for example (I like public speaking, most don't so that is not a universal solution). Have me write some code with you, if you must. I work at a very high level, but I am not necessarily 'clever' at seeing the right trick in 10 seconds under pressure in an interview.

Also, how important is that sort of thing, really? For some jobs it is, undoubtedly. If you want me to write a load balancer for your cloud infrastructure, there are some algorithms I need to understand. Even then, not knowing something now is not evidence I can't do the job. I can learn - it's how I've done everything up to now. I can learn your thing too. More importantly (to my mind) - am I a hard worker? Can I mentor others? Can I lead a team? Can I put a schedule together, and recognize when that is appropriate and not appropriate? Can I make a budget? Can I talk to clients? Do I meet my deadlines? Do I solve problems - not implement the cards in this sprint - but see friction of some sort at work and engineer a solution? Do my peers like working with me? Do I get shit done? Is my code maintainable, readable, modular, commented? Can I write? Can I teach? Can I work with the CEO?

I've been to so many interviews where they don't even try to learn any of that latter stuff, where they show no interest in what I have done, where they don't try to figure out if I can do the actual job they are doing, but are deeply interested in if I can program a smart pointer, from scratch, to some arbitrary set of requirements. It is all just deeply puzzling to me.

I understand what you mean about ridiculous SV interview practices. And your point that all we should really be measuring is on the job performance is an important one.

All I'm really going for is to ask someone to write some code live, on their laptop with their tools, so I can watch how they debug, how they think through things, etc. I want the problem to be hard enough that the person has to model execution in their head and that it's challenging to do so.

I have found that there are people who can code decently well but get badly stuck when the complexity hits a certain level. I'm trying to not hire them in senior positions.

For example, I had a recent project where we were analyzing user data and producing a bunch of edges representing potential connections detected between users, as a way of sniffing out fraud rings.

One feature associated with this was a banning UI, allowing loss prevention types to navigate through the graph, select groups of users according to certain criteria, and generally look for patterns that are evidence of fraud rings manually, to supplement the automatically generated analysis.

There are a lot of front end people who will go into brainlock on this kind of thing. I need to make sure I have some people around who don't.

When making senior hires, I want people who can write code for this sort of thing without making a huge mess or just getting stuck for weeks.

Presenting a real sample from a problem like this needs way too much context though.

I'm going to keep thinking about how to precisely define the type of task that i'm really talking about here. My fuzzy definition about ability to model execution of X complexity in one's head is obviously weak and probably flat out wrong.

The best questions have levels. Speaking to recursion specifically I had Flood Fill as a question the last time I interviewed. I solved it with recursion as it's the more ... "elegant" method. Then the interviewer asked about other ways to solve it, which went to iteration. After which we could briefly chat about stacks, bounds checking, etc.

So when it comes to "junior, normal, senior" they dont have to be different questions, in my opinion. Opened ended questions give flexibility for the candidates inclination and complexity in answering. Depending on how the candidates doing add increasing complexity, discuss improvements, or try again with a simpler setup. Something like Word Count can go many different ways; If they like C talk about char and buffers and syscalls. If they like javascript, well then you can make fun of their tail call recursion.

I did not get trick questions, which is good. The focus was on if we were a good fit for each other. I wanted to know as much detail as possible about projects I could begin work on, to see how much value I could bring, and to confirm that I could perform. We spent a lot of time discussing that.

Netflix does like to hire senior staff, who often have previous public accomplishments they can refer to, and discuss in interviews.

Brendan, I'm interested in your reaction to what a few other posters wrote and floated to the top regarding very aggressive, tactless hiring manager gleefully indulging about firing people or brow beating about quick firings due to performance, seems several people have the exact same experience and it belies your post a bit.

I was surprised, but the original poster followed up to say that that person has left the company. Would be helpful if people can note when the bad experiences happened.

I got some blunt answers (reinforcing what was in the culture deck), which felt different than the happy-talk you can get with other companies, but I appreciated the honesty. I didn't get anything tactless.

I'm a recruiter and have worked at a few startups. My wife is looking for a new role and recently had an interview where they sent her questions and asked her to record answers and send them back. This blew my mind. What a horrible experience.

For me, the interview is absolutely 2 ways. I spend just as much time answering questions and talking about our team as I do asking questions. There are times where I walk away wishing I had done a better job, but you always HAVE to care about candidate experience.

Customer satisfaction is disproportionately tied to the sales experience. I still resent the guys who sold me my Toyota. It stands to reason that how we are recruited also directly impacts how we feel about the company after we join.

I'm still angry about about a 96 Jeep I bought new. A little untruth by the sales guy about a minor feature, that was important to me, has turned me off all jeep products ever since. Nearly twenty years later I cannot look at any jeep one without remembering that profound disappointment when my new vehicle was not as I had expected.

I've been beating this drum for a bit now at my company. My employer is extremely prestigious for it's main field but largely unknown in the software engineering world. It doesn't help that we're literally across the street from a handful of household name level software companies. And yet, a common complaint (I had it myself when I joined) was that people feel put off by the hiring process that we have. To make it worse, I know for a fact that this sentiment is spreading via word of mouth locally.

My point to the folks at my company is that we need to be improving our brand in the eyes of software professionals, not making it worse. To do this we need to present an image that's not just bog standard, mid-sized bureaucracy style hiring.

Have to agree with this. I've turned down offers, or stopped the process, due to a bad interview experience. It's not a good signal, necessarily, but it is the only one that we have, as to what it might be like to work for you. If you can't get this together, what else do you do badly? I recognize from all these comments that companies can be quite nice internally but just suck at interviews, but what else do you have to go on. If my potential future boss finds his phone more interesting than talking to me during the interview for example, why would I accept that job offer, especially in today's market when my phone is literally pinging with interview opportunities as I interview with you?

And I that perhaps a bit more subtly than it appears. This frantic search for warm bodies can make you feel pretty desirable and important, and I think it is easy for that to skew your thinking a bit and to over-interpret small things that happen during interviews. But, that is the reality of human behavior, and you cannot ignore it.

What other big companies have awful candidate experiences?

I recently went through a job hunt and two stand out in my mind: Red Hat and Rackspace.

Neither one of them insulted me or anything, per se. Rather, I saw that they were looking for engineering people on a couple different job boards and instructed you to apply through their 3rd party system. I had done a lot of work lately in virtualization and distributed computing and these guys are good open source citizens there, so I figured I'd apply. Two months have gone by now and still no word -- my status on their website remains "resume received".

Makes me wonder how they ever manage to hire anyone at all. It's really soured me both to them and companies that use that shitty application system in general. At this point I figure anyone who uses it (including Amazon and Oracle) aren't even worth wasting my time messing around with.

The reality is that most job postings aren't tied to actual jobs that need to be filled right now. These postings just hang out there on websites and passively gather resumes. Occasionally someone actually looks at the applicant pool for 10 minutes and picks a few. The rest are never responded to or dealt with in any way. You'll probably never get a response, or you'll get one three months after you've started another job.

My personal favorite experience was being called (on my desk phone) by a recruiter in response to an application that I submitted, for a different job at the same company. I had applied 4-5 months previously, and had been working there for at least 3 months. One hand seriously has no idea what the other hand is up to.

I sympathize with you, this can feel pretty bad as an applicant (especially if you've done a lot of work on your resume / cover letter just for that job). Once I realized how things work on the back end, I stopped caring or even thinking about jobs until an actual human reached out to me.

Current rackspace employee - ime, we want you to apply through our system (here http://rackspace.jobs/). If you're getting no reply from us at all, then there's a good chance that no one here has got your CV.

(Or the US does thing entirely differently - I only work in the UK business unit, so you never know)

That is the system I used.

I'm sure no one actually got/read my CV. It irritates me to no end that I can't talk to an actual hiring manager, can't provide a cover letter, etc.

Hiring is a two-way street. The candidate is obviously trying to sell himself or herself, but the company should do the same. Some companies like they are doing you favor by hiring you. I cannot imagine they get great talent.

I think a lot of candidates also forget this, partly because they "need a job".

Reply on news.ycombinator.com