Hiring Is Broken?
software.rajivprab.com
software.rajivprab.com
My personal experience interviewing is that people cannot easily glamorize their past projects when pressed for details. It might be easy to make it look good on a resume, but hard in person during a discussion with follow up questions.
I think asking about past projects is a fantastic interview question, and it’s very easy to get a good sense of what they did, what they were trying to do, how much was their leadership versus others, how successful it was, how much effort it took. All you have to do is ask questions about it, and not take the resume item at face value.
Personally, I find the open ended nature of the past projects question to be a better predictor of future performance than almost everything else on that list.
Anyone who wants a job knows it’s a bad idea to lie because you find out immediately once you hire someone whether they are lying, and it would be grounds for immediate firing. Also follow-up questions weed out lies & half-truths very quickly. You should try it, see how long you can answer probing questions about someone else’s experience. I agree that most people won’t lie, and what does happen is almost always not lying but just spin, trying to glamorize rather than outright fabricate. My experience is that people get honest about their glamorizing very quickly if I keep asking more questions.
Do you really believe this? Lets say we have a mediocre person on a team who tells the story of him being a star performer. How would you catch that? He went on the design meetings, he know why every decision was made, he has worked on the project for years. There is no way you would catch it, and the halo effect would likely mean that you still think he is doing decent work but that the environment might have made him a bit worse but you wouldn't believe that he lied.
Eventually the liar runs out of those details when pressed.
Yes, absolutely. I’m not lying. :) Like I said, I have lots of experience with it. I’ve interviewed people who tried to inflate their contributions, and it’s easy to tell over the course of a 5-10 minute conversation.
> How would you catch that?
Easily, they would fail to be a star performer, and fail to match the abilities they advertised. You can’t lie your way into good programming.
However, it’s bigger than lie or not. Seasoned programmers have mature philosophies, and you can tell by talking to them. And honestly, I don’t mean this judgementally, but your question does sound like it comes from a lack of experience. You will find out what I’m talking about in time; big lies are not in the least bit easy to pull off, and almost nobody tries that kind of thing because the probability they’ll get caught and have serious consequences is very high. Being present in design meetings and knowing why decisions are made does not make you able to talk about it fluently.
BTW, while I believe this myself absolutely, I don’t know for absolute certainty that I’m right. So if it’s easy to do and the payoff is large then you should do it and write about it! If you could easily lie your way into a job, then it should be easy to work your way into one of those super high paying jobs, making robots or neural networks, and get away with it for years.
Agreed. There's so much more that goes into programming than just knowing a programming language or two, so even if someone does know basic coding or has medium understanding of a former coworker's acxonolishments, there's so much more in the domain that (in my experience) liars, fakers, and overhypers are exposed very quickly after hiring. Granted, understanding this doesn't help weed them out before the fact, but still I think most interviewees know this and are wary of spinning their stories too far out of control.
It doesn’t take long before people start stammering and deflecting if they’re full of shit.
The end result is that you will hire people for high level roles like tech leads or architect who are very mediocre at what they do since they sound like a star performer at the interview. All you need to get a good career in such an environment is just to job hop a lot and at every job you mostly try to gather impressive things people around you do in order to sound impressive in the next interview. I bet a no small fraction of CEO's went that route, and I think it is a pattern we should avoid since it is so easy to test actual coding skills in tech.
An example of such a question could be ... can you tell me about a time that you tracked down a bug that you at first thought was not possible but later identified and found a solution for?
Why would they? You’re making assumptions. Being in standup doesn’t mean you understand so well you can talk about it. Knowing why good decisions were made and actively making good decisions are two completely separate things.
Do you think you’d be prepared to be the lead singer or guitar player for a band if you memorized every note of every song of your favorite band? Would it help if you listened to all their podcasts, read all their books, studied their history? Would it help if you were their roadie or even their manager or producer? The answer is no, being near them and witnessing their output does not equip you to mimic their creative process, or to even understand it. Same goes for programming.
> Being able to walk through someone else’s though process is a lot easier than doing the actual work.
I’d say that’s true, and is exactly why you’d catch a liar in the unlikely event they made it through the interview process.
> All you need to get a good career [...] is just to job hop a lot
Yeah, it’s so easy. So do it! You have to keep lying and continue to convince every single company. You’d have to avoid people ever finding out and/or avoid companies calling each other for references. This takes so much effort, it’s literally easier to go to grad school and become an amazing programmer. What is making you think a lot of people are doing all this lying? Are there people around you that you suspect are lying?
> I bet a no small fraction of CEO’s went that route, and I think it is a pattern we should avoid since it is so easy to test actual coding skills in tech.
Multiple enormous problems with that sentence. CEOs don’t need coding skills, they need organizational skills, sales skill, communication skills and a host of other things, none of which are writing code unless you’re in a startup. (And then CEO is a goofy term anyway.)
You’re speculating. If a huge number of CEOs are faking it, then it should be easily demonstrable, so find some evidence. When CEOs fake it, the result is the company goes out of business.
It turns out we do avoid this “pattern” because it’s not really a pattern, it by and large isn’t happening, and there are many forces preventing it from happening that you’re not acknowledging.
Last but not least, coding skills in tech aren’t the most important. They’re very important, but not the most important. Teamwork, communication, attitude, and goals are all more important than actual coding skills.
If he worked on it and could convincingly fake it, I - as a recruiter - wouldn't care too much if he embellishes a bit and claims he also did it from the start. Maybe he factually didn't, but if he's good enough to make me think he did, there's a decent chance he would be good enough to actually do it for me. Not a certainty, there's never certainty, but a good chance. Of course, given the choice, I'd prefer the one who actually created it, but one who worked on it and knows it well enough is not bad either.
I would say it is OK if it works for you but it still might be that you drop off some good guys. Because it is environment that creates good employees, and if someone was working for months or years in a bad environment, he can be perfectly capable of doing work, but not being confident and feeling good about talking about his last job.
The coding interview itself is an immune system against the surprisingly large population of candidates who have long resumes and can talk a big game but not actually perform. (I didn't believe these people existed either, until I started doing phone screens).
It's also a way to recognize smart people who will do well with whatever they're given, even if they're underutilized or in a different domain at their current job. Tech companies are typically hiring more for this than for tech stack or business domain similarity.
You're trying to measure how they'll perform on your work before you hire them? That sounds like you're trying to get free work from them. (Just a note.. this is on the words you used)
> recognize smart people who will do well with whatever they're given,
How does a take home coding test that is limited on time, and their own time + stress indicate that they are in that category?
I wish the hyperbole about trying to get free work during an interview would just die already. Nobody is getting free work by interviewing applicants, interviews are costly to most companies, but necessary to grow the team.
Just because interviews take time and are difficult and scary doesn’t mean someone is taking advantage of you. Try to put yourself in the company’s shoes — they have to evaluate you one way or the other. How would you do it?
A time limited stress test is extremely effective at identifying certain kinds of performers. Obviously being able to solve problems fast is an indicator of a certain kind of smart. This doesn’t identify all smart people, and nobody claimed it did, that’s why a coding test is almost always one part of many in the interview process, and people routinely get hired when passing some parts without passing all the parts.
Most companies have 4 sessions : 1 HR phone, 1 tech phone, 1 tech onsite, 1 HR onsite.
But bigger companies (i.e. bigger salary) increase the 'tech onsite' up to 4. Some companies replace one of the tech phone/onsites with a take-home challenge.
If you want more money, you gotta beat the other 100 people who also want more money. Simple case of demand and supply.
Which is true. Companies know this.
Most of the stuff I read here is basically made up of people who aren't quite the best moaning that they couldn't get a job at Google or whatever. Life goes on. It is impractical and not a good thing for every software developer, even the very good ones, to pile into the same companies.
Not everything is set up as a perfect procedure for you.
And even if it was - OK, so you get the job and the Stanford grad doesn't. You swap places. Does that make a better world?
I encourage new developers reading these sort of posts to just ignore the fluff and plug on. You'll get there.
And if not? In five years you might want something else entirely.
I wanted desperately to work at an investment bank as a new grad.
I'm very happy with the path my life has taken instead.
Many of these "hiring is broken in tech" posts are like you said: moaning or a remarkable lack of self-awareness.
There are a trillion arbitrary signals that will help or hinder your success in various ways.
Physical attractiveness, gender, race, parental background, your nationality, where you can work, your accent, language... the list is honestly almost endless.
You can wish for a world in which none of this were true, or do your best to find a place within it. Sure, don't lose sight of justice, but recognise that you can't change the world alone.
For all of the reasons I stated above?
I think you'd have to be severely mentally deficient in order to think that commercial software development operates in some sort of strict high score board fashion.
OSS development I think is significantly closer if you allow for the sampling bias that only certain groups have enough time to contribute.
Some of it is like that, but not much.
As much as I like open source, there is a lot of myth making around it too.
________
Second whether obvious or not, people argued that we meritocracy on these forums here for years. And some still do. At meritocracy peek, Github had even that meritocracy carpet or whatever it was.
Anyway, FOSS makes yearly polls about these topics. It is biased toward more involved projects, but again I don't think someone's one weekend demo is what is talked about here.
As someone who has spent more time on the hiring side than the job seeker side over the last 10 years, I find these articles funny. I ask myself if each method could tell me what I need to know about a candidate. For many, the answer is no. Then I ask myself if I think the remaining method are fair and often the answer is no. Then I think about how much hassle they are for the candidate and the best methods seem really onerous for the candidate.
I always come back to the way I handle it. And now that I'm looking for a job again I had one interview where the technique impressed me enough to change my style a bit in the future, or at least add some options to what is a dynamic process.
Perhaps you're speaking from the perspective of dev jobs in Europe, or just outside the US, but even in the midwest where the only "tech" companies are really just the largest commerce/healthcare/financial institutions, dev jobs still pay $75,000+ for entry-level positions. Ignoring the fact that cost of living and taxes are way lower in the midwest than they are on the coasts, it's still a fact that with just a bachelors degree computer science opens up a plethora of opportunities.
If the price to pay is grinding Leetcode problems for a few weeks that's a pretty small price to pay. Honestly.
And the commerce/healthcare/financial enterprises that you mentioned are really picky and don't even care about leetcode, they filter based on resume prestige like law firms. Without connections, having attended a prestigious University or experience at other fancy companies you ain't getting your foot in the door regardless of your leetcode level.
If you've only been coding at mom & pop corner shops, you'll be doing that for a long time as most big enetrprises you mentioned won't even look at your resume.
There is no proven method for sorting a pile of candidates such that the absolute best engineers will float to the top. In the case of high-profile employers, a significant percentage of hires will be those who have trained themselves specifically to beat the hiring process at that company. Those people aren’t the absolute best software engineers.
Which is what people talk about here.
No-one ever mentions the fact that say, you're barred from cracking out `git clone` and making the next Linux.
It's always commercial employment, which always has been and I am willing to bet always will be primarily a political field.
Obviously Google doesn't have the best devs - or at least I bloody well hope they don't - because they're mostly working on ad conversion shite. That excludes a ton of people I know right off the bat.
There’s no proof that they are but there’s also no proof that they aren’t
[for the sake of argument] -- yes, there is a method, if you want to find out the top few best engineers. Look at their experience. Jeff Dean, Linus, John Carmak. I think my algorithm fulfills the letter of your search criteria by finding a few 10X engineers in an ocean of regular people.
Unfortunately, the 10X (or arguably, 100X in this case) engineers may not be enough or not be interested in the job opening you have.
So many companies I would consider mid-tier, lower, or not even technology companies have outsourced have adopted similar if not the same hiring models of FAANG or outsource hiring/recruiting/assessments that follow suit. It's an absolute mess.
- "interviewing is broken" is a meaningless linkbait title. All you can really say is it's less efficient than it could be.
- In fact, interviewing in tech might be the best of any industry. Look how good we are at hiring minorities, people without college degrees, people who don't wear suits, people with verbal tics, or even tattoos. That wouldn't happen in investment banking.
- If your point IS that hiring could be better, then instead of winging, propose a fast and simpler and cost-effective alternative, and then try to give some hard evidence that the method you propose is better than whatever you're winging about (usually leet code).
Wait, what? Are you talking about the software industry?
One of my most popular HN posts [1] is about how the job search sucks so I empathize a lot. However, I have little interest in working for “top” companies.
Or it’s a bunch of very competent coders annoyed that they’re made to jump through hoops.
Effectively they simply select for people willing to hack their hiring process, which also excludes most actually talented developers. Resulting in a few very talented people, and an overall average workforce.
This is also why most internal projects at these companies are about average for the industry and these companies focus so heavily on acquisitions. Consider the differences between iOS, Android, and Windows phones was not so much about software quality as much as business models.
Fundamentally, companies do not hire for raw technical ability.
Would you hire Linus for your bog standard dev role? Of course not. He'd be bored in days.
You probably wouldn't want a supermodel as a life partner.
The whole thing is about selecting model employees. People who will rock the boat in the correct ways; need money, but are neither impoverished nor independent; are reliable; and so on and so forth.
It's not "hiring is broken". It's "you misunderstand what employment is".
He was a great resource but hiring him permanently would be a terrible mistake for all parties.
I've often wondered if Rob Pike or Guido Van Rossum went through the same interview process at Google that we all hear stories about. I'd be very surprised if they did and my guess is, for them, it was much more of a negotiation of what they were going to work on and what their compensation was going to be.
But then, if you ran a business, that's almost precisely what you'd do. It's completely rational. The hoi polloi need to be filtered aggressively, most companies need to select for hunger, etc.
By contrast, you might as well give a big name an honorary position if you can afford it. If they do useful stuff - bonus. Hell, if I owned a company the size of Google, I'd give Guido some money just because I could.
The next time a recruiter from there cold-contacted me, I asked again. IIRC, that one said a person could skip at most only the first part of the whiteboard coding hazing (phrased differently), if someone at Google could vouch for them, but that everyone had to do the second part of the hazing, before they could advance to the interviewing phase of professionals with mutual respect (phrased differently). I decided not to interview.
So long as their process comes off as one-sided, arrogant, and negging, the company doesn't seem like a place I'd want to work.
It's like you're meeting a first date in the line at a cafe, and they're physically attractive and rich, but they're rude to the barista -- bullet dodged, we're not together, and I'd like my coffee to-go, please.
Being asked to perform these tasks is actually a pretty good litmus test to filter out companies. Also, I expect a certain degree of deference given my resume shows a 25 year history of implementing and delivering systems.
There were applicants with a decade or more of experience who, like you, expected some deference and were annoyed or offended, and complained about having to take a coding test. But they were applying for programming roles, not management. Those people didn’t get hired; negative attitudes are a red flag. In effect, this particular company likewise used them as a litmus test to filter out applicants unwilling (or unable) to demonstrate their advertised abilities.
Note especially how the coding test is actually being used to evaluate attitude and things other than coding skill. It’s not valid to assume that being asked to do a coding test implies that someone doesn’t believe you about your experience.
That said, resumes are very easy to fluff up with big sounding projects, so talking through past project details and demonstrating your abilities with a coding test should be par for the course if you want to change companies.
I actually agree and I don’t think coding projects are necessary for hiring, but I do think they’re useful, not bad, and I don’t give deference to anyone when I’m interviewing, no matter how accomplished. I look for people who are smart, willing to learn, excited to do the job, excited to interview for my company, and willing to try.
When you're focused on system implementation, delivery and maintenance you can pick one of the past projects listed and start probing. It's not hard to quickly get a picture of what the candidate did and didn't do on the project and whether or not they're blowing smoke.
I'm happy to have real technical discussions and sketch out things, but I've seen enough negging "coding interviews" administered by smug people, and it seems to have little-to-nothing to do with legitimately assessing competence.
I'd also like to have an opportunity to get a sense of the people from talking with them. The FAANG coding theatre performance that undergrads now drill for doesn't tell me much about the team's personality, competence, and working style -- all it tells me is that apparently they are willing to play along with that dysfunctional one-sided game.
The question is incorrect. By hiring Linus you eliminate the need for 5 bog standard dev roles. Bog standard dev roles is not a requirement. It's what you end up with if you hire mediocre people.
Not everyone in FAANG are in charge of shaping the industry.
In fact, even with such hiring process, I would say FAANG problem is still having too many of such talented people but not enough internal growth so that they can't retain them.
You do have a point here which I do agree with which most of those companies would refer to as "culture fit". Agree with it or not, having an employee that gels with the team is a critical aspect to hire for. You could have 10 Jeff Deans and get nothing done if they refuse to work together.
A significant problem I see here is that we are trying to define "good" on a linear scale which makes no sense to me. Is candidate A better than candidate B based on 6 hours of interviews about the same topics or a single take home test. Even Madden breaks down a person into dozens of traits and uses that to compare people. Tech companies like to flaunt how data driven they are but why hasn't that made it into hiring. For the most part, these decisions are still made on personal feelings of the interviewers which...yeah, that's not going to be very consistent and reproducible.
Personally, I would place technical skill at around 50% of what makes up a talented software developer. Effective communication, ability and willingness to work with others, flexibly, work ethic, attention to detail, prioritization, etc are all important. But, I have found these things really tend to go together.
I'm glad that I'm not the best at what I do. I'd be working more hours, given too much responsibility, picking up the slack for people, subject to more meetings, etc. None of that is for me.
One thing is for sure, and that's that I'm the best me, and you're the best you.
I spent most of a day yesterday trying to get openssh 8.0 working on Debian 2 (with 1998 era kernel, glibc, etc). Because I can.
Genuinely more fun than all the toys money can buy.
It didn't even work. Fuck.
No offense, but this isn't good advice. You just contradicted yourself within two sentences.
Here's the "true" version of this statement:
You may not actually "get there". You may or may not want something else entirely, after that potentially traumatizing experience. You also may be six figures in debt now, good luck!
In software development, we have this tendency to not take things seriously. Career choices are some of the most serious choices you make.
I'm afraid we're luring all these otherwise uninterested people into this career path by promising them lucrative and future-proof jobs. What eventually happens is that an oversupply of underqualified candidates flood the market. Meanwhile, other trades that aren't always in the headlines are starving for workers.
Despite what you may want to believe, many companies are afraid of hiring sub-par developers, because the narrative is that they will eventually ruin you. We can argue about how true that is, but that is irrelevant to the outcome. What matters is the perception.
Not, however, if they stick dogmatically to the idea that it's Google or bust.
I'm not talking about a lucrative job; a middle class existence; a big fancy company; any of that shit. I'm saying that an intelligent human being needs to realise that the world is not necessarily set up for them and they either need to fit in, or carve their own path.
And about debt? Yeah, don't do that. The US college system looks like a racket to me.
Fair enough. A true scotsman will never put sugar on his porridge.
> I'm saying that an intelligent human being needs to realise that the world is not necessarily set up for them and they either need to fit in, or carve their own path.
Indeed, a human being must either do one thing, or, failing that, do another thing.
There's at least two ways of reading what you are saying. In the one reading, you are giving questionable encouragement. In the other reading, what you are saying is a tautology.
I assumed the former, but apparently you meant the latter. You were right all along.
It is indeed tautological.
Which is my primary complaint about the prevalent attitude displayed in these hiring posts.
Porridge isn't going to change its' recipe for you.
I don't like the term "culture fit" because it is a proxy for people who are already on the team, and often gets confused for "white, male, and likes to drink with coworkers." That said, I highly value "work style fit" and "team need." Are you running into code quality problems? Find someone who thinks about code quality. Do you need someone who can get a prototype to sales quickly to close a deal? That's a skill, and it's not a generalized skill.
All of the cases above can be valuable, but only contextually. Figure out what your team needs first, then craft your interview process around that.
(As for me? I have been in management for a while, but I am at my best in developing high functioning teams, collaborating between teams to solve larger problems, and crafting high quality solutions. I am not at my best in deep debugging sessions and quick prototyping. If it isn't my own startup, I might not be the ideal choice for employee 1-5. I may be the person you bring in when your team is expanding rapidly and you're running into scaling issues.)
If you are running an agency trying to drum up work, there are people better than me.
If you have a codebase with no tests and everyone is afraid to touch it, you want me. If you have three teams that are trying to sort out a problem but nobody will dig in, talk with everyone, and sort out the technical issue, you want me. That’s the difference.
Now, as for determining high quality code without having that basis, I would start with Robert Martin’s book “clean code.” At the very least, it can teach the basics of understanding a code sample. However, it can be a difficult problem. I might think about a consultant to judge those and use behavioral interviews to judge the rest.
> Ask them for references, Do back-channel references
Asking people for feedback, peer review, 360 review, etc. This is somewhat effective, but if people can choose their own reviewers, they will of course get only the good info, same as for interviews. If anyone can submit feedback, you're likely to get people who don't like an employee to write stuff about how terrible that employee is, which might be exaggerated.
> Take home projects, Audition for the role as a contractor, Ask them to describe their past projects, Ask for source code from significant projects, Pair programming on a “real” problem, Live Coding Exercises
All of these are meant to try to get real signal on how the person would perform their job. But even when people are employed on a team and you can observe them at any time, it's hard to quantify their performance. Do you look at their commit count? Bug count? Lines of code? Any metric has its weakness, and people game them.
Interviewing is a game of making a decision with incomplete information. I would say we aren't much better having a whole lot more information about how someone works, when trying to give an accurate performance review.
If we can't accurately judge performance of the people we should know, I think that proves we can't accurately judge performance of people we don't know.
I really don't think we're ever going to find a perfect hiring method. You will always have a nonzero chance of false positives and false negatives, and there will always be downsides that piss people off. So, pick a reliable hiring method(s) (talk about experience, solve whiteboard q's, do take-home projects, or whatever) and try to make it better over time. If you don't like it, you can switch to another method and start improving on that. Etc. But your method will always fail some good hires and pass some bad ones. It sucks, but we are still able to hire mostly good/trainable people, so we should stop worrying and move on.
I hypothesize that there are a lot more interviewees than interviewers getting involved in the debate. I only have to tell the former group to keep trying and stop complaining. If you're not sure why you're failing interviews, that's a solved problem -- just do mock interviews with your friends or on interviewing.io. But stop blaming the process for your own failures.
Wow, you're actually training people? Kudos to you, I haven't seen that in a looong time.
In my current area, everyone only looks for seniors or experienced devs. New grads without connections can suck it basically.
We hired a mostly self-taught engineer out of college a year ago (I started ~2mos ago). Had almost no idea about webdev, or C#/.Net, or web APIs, but we taught her on the job. I'm similar, since I had no experience with web APIs before joining but I had experience with C#.Net, Asp, etc.
If someone found a foolproof method of hiring once it become widely known, it would no longer foolproof. Either employers would all try to hire the same employees or employees would find out which signals mattered and tried to game them. This already occurs to some extent with the Ivy League hiring approach. The general “college degree” part has already been over-gamed to the extent it is meaningless.
What employers should be doing is measuring the long term success and failures from interviews and seeing if the interviewer even has an above random chance of selecting good employees. If it’s worse than random than the interviewer is probably just terrible at what they are doing.
This would also require the employer to agree on what makes a long term successful employee, something I’m not sure many people even agree on. At minimum, I’m presuming you couldn’t make a judgement on a hire in less than 5 years.
> If you solve the problem, you’re scored highly. If you don’t, you’re scored poorly.
In my own opinion, this is a terrible way to look at these problems, though many people do use this metric. This is further emphasized by HackerRank-style exercises where you have to pass all test cases. This is robotic and not empathic at all.
A more fluid and dynamic approach that focuses on the candidate’s approach, ability to clarify, and problem solving, even if the correct answer isn’t reached, can still give valuable signals
Disclaimer: I work and give interviews at a large tech co that uses exercises.
Maybe do it Survivor style, have 12 candidates/new grads start, give them the goal of producing an MVP CRUD app, and start voting people out of the scrum after a couple of days. Winner gets a 6 month contract to hire, 3 people have their careers ruined, and the rest bump around the valley for a few years before moving back to wherever the hell they came from. The memes that come out of this alone could make this show a valuable cultural contribution.
Without a flawless performance it’s unlikely for most people to get offers.
I am more looking into how a candidate thinks than if they get the problem correct.
It is more telling when someone is able to logically work through a problem they never seen before even if it isn't optimal or even correct.
I am more skeptical of those who blow through a hard problem getting the optimal solution right away. It likely means they memorized it or had lots of practice on such type of problems. I don't hold it against them, but often have to throw in some curve ball follow ups to see if they really know there stuff.
I'm currently interviewing, and I have to say I've taken enough pass/fail coding tests to start to shy away from coding tests, especially if they don't tell me what to expect.
I was a relatively junior person working there in my early-to-mid 20s. It was common practice for me and my peers to be asked to review resumes and filter them, as well as be an interviewer every few months. I never once received any kind of training on interviewing or resume review. Not even the 10-minute-cheesy-corporate-video kind of training. Hell, not even a quick 1-pager or chat on tips or things to look for. It was literally just "hey here's a stack of resumes, please filter them".
Of course, the difference is that the feedback loop is at least two to three orders of magnitude faster for writing code than for figuring out if you did a good job hiring.
(I'd be happy to train your team if you don't want to go through their archive.)
How do you know if you hired well? Is this an excellent result or a terrible one? How do you know how well each of those 5 did and how well the team would have done with other people in the roles?
1. Main thrust is that all the HN posts on this topic are stupid and formulaic
2. Argues that numerous the OTHER posts merely list the negatives for one method of interviewing A, and the positives for another method B.
3. Lists every method of interviewing, showing glaring problems and advantages in each of them. Obviating the need for the discussion ever happening again.
4. Asks for something akin to evidence if we are going to drive the discussion forward.
If the focus was to be on evidence, it would provide a framework for such a study. It generally sounds pretty hard to do anything of the sort.
Uh, what? It start by saying "I can’t keep track of the number of articles I’ve read about hiring in the past few years. Inevitably, they all follow the exact same format"
That point is true.
The question then becomes "Why does the community generate spammy articles without making any sincere attempt at intellectual progress toward a better system?"
And the answer is probably that it's not in any individual's self-interest. HN members want to whine about not getting hired by google, companies want to promote themselves, people like me want validation.
The only people who have any incentive/interest in objective hiring are companies that specialize in that purpose (e.g. hired, interviewing.io). But there's really no reason for them to divulge their secret sauce, and even if they did, people would probably only upvote it if it let them feel less insecure about getting rejected by google.
Hiring might be broken for keeping out the best of the best, but as long as you work at a company that isn't afraid of firing/laying off, hiring isn't fully broken.
Yes, it is--it is just broken in a way that privileges the company over its employees. Firing somebody (absent flagrant abuse and the like) is a failure of management and leadership.
There are lots of good schools. If you got a BS in Computer Science from UIUC, that means something to me. That’s a good school and you got through it.
Same with jobs, why does “Look at their job history” mean “Check if they went to any big name tech companies”?
You look at the companies, look at what the company does, what the applicant said they did for them, and ask them questions about what that was like. It doesn’t really matter if it’s Google or it’s Pet Food Express, the important questions are the role and the work.
I think this is what TFA is getting at.
As a basic example, in a lot of places you get very little advance warning that you're going to interview someone.
Test problems or assignments are often not tested or even looked at by existing employees, so often you don't even know that your current team would do well on them.
Sometimes interviews happen without any intention at all to hire the candidate (wtf).
I realize this doesn't scale to the rapid growth of some companies, but most of us need a more modest supply of talent vs. giant pools of people.
Assuming you had some way of making comparisons across these different formats (easier said than done, I'm sure) it seems like you would end up with "truer" signals. It also seems like you would get an extra signal based on what the interviewee chooses. For example someone who prefers live coding/pair programming sessions might be someone who leans on and thrives more in ultra-collaborative/brainstorm-ish settings. Whereas someone who prefers take home assignments maybe is more productive being able to sit on a problem and iterate, then collaborating after being able to articulate the specifics in a code review type setting. Neither are wrong, nor are they mutually exclusive within an organization but in terms of team placement or general "culture fit" (whatever that means) it seems like something that's useful to know ahead of time.
Of course you might also allow people to choose which route they think they can hack more but I think that problem is more of an implementation detail of any given method. Asking for a solution to find the perimeter of a group of pixels at a video streaming company is better than asking someone to invert a binary tree at virtually any company. Asking someone to build something real-ish with a hundred valid solutions is better than asking someone to implement a trie with a thin search interface on top. And so on.
basketball analogy time! (my favorite =) the NBA employs 450 of the best basketball players on the planet, and at any given time, the league is evaluating a few thousand more (out of hundreds of millions of basketball players, out of ~7.5 billion people worldwide). all the easy filtering is done. every year, the 60 most promising players are drafted. and because they're among the most promising, most of these 60 players should become regular NBA players. in practice, only 5-10 will do so in any given year, and you have no idea which ones it will be as they all look promising (e.g., laker kyle kuzma drafted 27th in 2017).
in short, once you get down to the 10-20 candidates you think will fit a given job, you might as well just draw a name out of a hat, work with them for 3-6 months, and re-evaluate. there really is no shortcut to this (not even for google).
going back to basketball, no team has yet to sign 10x nba all-star carmelo anthony this year, and 8x all-star dwight howard only just signed with the lakers (on an unguaranteed minimum contract) because of a season-ending injury to demarcus cousins. both players are individually very talented, but are reputed to be hard to play with (thus lowering team productivity & efficiency).
Professional lifetimes are also frequently quite short. For gymnasts, competing life is virtually over by age of majority, for Americna football, a few years in pros is typical, though some careers last longer. Baseball, cricket, tennis, and golf are longer-lived exceptions.
I'm not saying this as a rabid sport ethusiast, and the transfers to technical recruiting may be limited, but it's at least an interesting system to consider.
Other uneven talent/fame domains are theatrical performance (acting, music), and, possibly, writing.
so then, why would one expect to be able to evaluate a candidate for a few hours and do better than that? if anything, professional (office) skills are harder to evaluate than athletic ones.
reasonable folks should expect only 10-20% of hires to be good fits, regardless of the skill or integrity of the hire.
1. A verbal pop quiz about some reasonably-complex NetSuite-related programming topic/task (i.e. "on a high level, how would you go about implementing $FOO, given the constraints $BAR and $BAZ?")
2. Digging into past projects/experience and gleaning info on how involved the candidate was with those projects
3. If available, look at an existing programming portfolio / GitHub account / etc.
I feel like these three things tend to cover my bases, knowing full well that they're flawed assessments (but flawed in ways I can control and adjust/tweak as necessary).
How about an accountant to do your taxes?
Further, how do you define who is the best? Someone who does a so-so job in a short amount of time? Someone who checks every possible decision branch, explicit or implicit? Someone who writes code in a clever way.
You're only ever going to see the skills you understand. Anything that you do not understand you will miss.
They'd be more useful in a buyer's market.
What you'll end up with is people who are desperate for your job, which may or may not be a great sign.
- Should not take longer than 3 hours for an inexperienced developer.
- No learnable skills tested like tooling, language, and domain experience.
- Simple to understand problem
- Problem should be obviously mappable to a solution
- Provide sample input and sample output data (only 1 case).
- Did the code work on the sample I provided?
- Were edge cases in the spec accounted for?
- Has the engineer followed a language standard (PEP/PSR)?
- Were docs (install, compile, comments) provided?
- Were unit tests provided?
- Were sample cases the engineer ran manually provided?
- Was the code "elegant"? Could another engineer at the company look at the code and maintain it in a year.
Is it just a single "write a method that does this" type problem? I ask because I did a take home very recently which I know I could knock out quickly, but it took much long because everything you listed ballooned the amount of time it took me to complete it. Specifically:
- Has the engineer followed a language standard (PEP/PSR)?
- Were unit tests provided?
- Were sample cases the engineer ran manually provided?
- Was the code "elegant"? Could another engineer at the company look at the code and maintain it in a year.
Lastly, this isn't mentioned but I did this on the take home I recently completed - using version control and properly branching & merging on each deliverable. Doesn't take time, but if you are making atomic commits with good comments, that's just another way for you to lose time. Of course, I don't think the fact that I did this is going to be noticed as it wasn't written out as a requirement.
If you notice what I'm nitpicking, it's all the setup and writing that becomes a time sink for most. In isolation, it wouldn't break the bank, but with all those requirements combined, you've lost much more than 3 hrs. When I'm given take homes for interviews, this is where I've burnt the most time - taking the time to document well.
You could check if she would think of the required parts. For instance, would she think of unit tests? If she wouldn’t mention tests, you could ask „How would you ensure that your program works as expected?“
Thus, solving the take home theoretically.
At a previous company we gave a problem that boiled down to the following:
1. Read a config file that contains a list of "Rules"
2. Each rule contains a name, an ID, a comparison, and a value.
3. Read a json array of numbers from a file.
4. Apply the rules you read to the array from the file
{
id: 1,
type: greater-than,
value: 10,
name: Number was passed threshold,
}
The instructions only specified one requirement. Something like:
Provide a method for our engineers to run your program
via command line:
`./<your program name> <rules-file> <data-file>
For your code to be formatted and for you to configure a linter are two separate goals. For JS (were standards vary greatly from org to org) I don't mind as long as there is a consistent formatting. Any IDE or editor has a format function that I would be happy to see used.
> If what you ask is of sufficient complexity
I hope that it is evident that this task was designed to not be complex for anyone but the most junior of developer.
> Of course, I don't think the fact that I did this is going to be noticed as it wasn't written out as a requirement
This sort of thing is exactly what I am interested in with this homework. You were given requirements and you reached for a set of tools that you thought were helpful to you as an engineer despite them not being required.
> If you notice what I'm nitpicking, it's all the setup and writing that becomes a time sink for most. In isolation, it wouldn't break the bank, but with all those requirements combined, you've lost much more than 3 hrs. When I'm given take homes for interviews, this is where I've burnt the most time - taking the time to document well.
I don't hold a lack of documentation or extra items against a candidate. That is made clear early on. There are tools that are very simple to setup or things you can do that would make it obvious to me that you know what you're doing.
You don't need a linter to format your code. You don't need a unit test framework to write unit tests:
const apply = require('./submission').apply;
const tests = [
{rules: [], data: [], output: []}
];
tests.forEach(function (test) {
const output = apply(test.rules, test.data);
if (output != test.output) {
console.log("Failure");
}
});
1. Code is obviously not working
2. Code doesn't handle my sample data
3. Code doesn't handle an obvious edge case (no data, for example)
4. Code doesn't follow the one requirement (./<program> <config> <data>)
Internal company standards make linters, testing, documentation, etc a "teachable" skill. I don't care if you do that on your own time because when you come to work for the company you will be doing what the company thinks is a good idea for it's source code.
(edit, forgot the "multiple" rule)
Restrictions:
- Should not take longer than 3 hours for an inexperienced developer.
- No learnable skills tested like tooling, language, and domain experience.
- Simple to understand problem
- Problem should be obviously mappable to a solution
- Provide sample input and sample output data (only 1 case).
Scoring:
- Did the code work on the sample I provided?
- Were edge cases in the spec accounted for?
- Has the engineer followed a language standard (PEP/PSR)?
- Were docs (install, compile, comments) provided?
- Were unit tests provided?
- Were sample cases the engineer ran manually provided?
- Was the code "elegant"? Could another engineer at the company look at the code and maintain it in a year.
Otherwise it's a total waste of everyone's time.
Extra points if you have a tweed vest and a smoking pipe.
Group A: People who excel at face to face interviews(extroverted) will complain that it is waste of time and disrespectful, because it demands significant time commitment compared to a 1 hour in-person interview.
Group B: People with interview anxiety(introverted) will excel at this type of test because they have the freedom to set aside their own time, do things at their own pace and environment.
However, people from Group-A has a point, because certain take-home assingment are notoriously complex and will take multiple days of effort if you want to meet all requirements, which are not an option for people with family & existing job.
I don't think I've ever been in a software engineer interview where the face to face portion is less than 3 hours (at minimum!)
My problem with take home stuff is that it's asymmetrical. That is, if we're doing a phone interview, it takes my time, and it takes your time. Ditto an in-person interview. You're serious enough to commit some of your time to it, so I'm willing to also do so. I know you're looking at other candidates, but if you call me in for an in-person interview, I figure I've got at least a 30% chance of landing the job. That's enough for me to be willing to invest the time.
But with a take-home assignment, it costs me much more time than it costs you. So you could be giving it out to tons of people. How many? I have no way of knowing. So you're asking me for 6 hours (which may turn into 12), but what are my odds on getting the job? Are they still 30%? Or are they now 1%, because you just gave this assignment to 100 people?
I'm not going to plow 12 hours of my time into a 1% chance at a job. Just no. And the problem is, if you tell me "oh, we're only looking at a couple of people", I have no reason to believe you. But if you're actually interviewing me, I believe you, because it's too expensive for you to interview 100 people to hire only one.
I'd support more complicated take home skill assessment tests later on in the recruitment pipeline, as long as they add real value for both the candidate and the company, and they're only given to well qualified and screened candidates. I'd also be concerned at a twelve hour test - that sounds like something very poorly scoped.
In my experience, they provide a great way of understanding a candidate's skills. It also lets them work in a low-pressure environment with their own tools, which tends to let people achieve their best.
I actually recently launched a service to help make managing take home tests through git more efficient: https://candidatecode.com
It's still early stage, so any feedback is appreciated
The key takeaway is that rejection hurts emotionally. Programmers who care about coding, take the rejection personally. Less time commitment, less hurt.
Still requires work and a subjective estimation of candidate skill because you have to make a judgment across metrics instead of within one, but I mean, everybody wins in this case, right?
If you don't had that scale, uncommon tracks will be full of interviewers who are out of practice for your question, and poorly calibrated to evaluate the candidate.
Also, some companies might not want to allow any of these methods, since they each have their own tradeoffs/blind spots.
https://techcrunch.com/2015/03/08/on-secretly-terrible-engin...
So, why not randomly pick 5 of these practices and simply tailor your process so it works well for a majority of candidates?
Something like pick your poison?
What else are you supposed to say about yourself or anyone else?
I'm not talking about the best developers in general, I'm talking about "the best developer in the team". The "golden child". Do you sincerely believe they're thrilled about giving up that special status to the new hire?
Yes, people are that petty, they're not necessarily that self-aware though.
Most people are fine, and if you run into a toxicity then go take care of it like an adult and talk to your manager/HR/whatever.
If you follow the advice of the GP, you are setting up a conflict of interest. I'm not ruling out that the "best developer" in some teams is of such noble disposition that they will always hire someone better than themselves. It's not what you can expect though, so don't set yourself up for failure.
I LOVE IT!
*ample use of scare quotes because I found "different developers work well in different situations or best with certain people" to be more true than "there's this 'best' developer let's clone them".
There is an alternative, use an agency. The problem with that is that companies will simply reject the proposed candidates without justification because hey, they are the customer.
However, it's worth a read because it does a good job of summarising the trade-offs inherent in most of the techniques I've seen used to hire engineers. I might have missed it, but it didn't talk about any kind of software design assessment, which I have used and seen used reasonably successfully.
There was, however, one observation that ticked me off under cons in the "Live Coding Exercises" section:
> Very stressful. Many programmers can’t handle the stress of coding under the clock, with someone watching them
If this is you I'm going to give you some advice that, if you can swallow it, will prove very helpful...
Grow up.
Seriously: grow up. You are (or will be) highly paid, and are supposedly a professional. So learn to toughen up and handle the pressure.
Because I guarantee you this: you will grow in any job that is worth having. Sometimes that growth will be forced. Even when it's not, it can still be uncomfortable and stressful.
Why?
Because you're doing things you haven't had the chance to become comfortable with, or there's some external pressure: deadlines loom; a teammate is off sick; a horrific security flaw is discovered in your code, or in a library you use; some terrible bug makes it into production (this happens at even the best companies) and you need to hustle to fix it; politics happens, and you have to navigate through it.
In this line of work you cannot avoid stress. You can certainly make every effort to run an orderly software development process with great teams, regular releases, great feedback mechanisms, interesting work, and all the rest, to minimise chaos, but that doesn't get rid of the stress.
We're software developers. Our stock in trade is dealing with really hard problems. Sometimes those problems are technical, sometimes organisational, sometimes commercial, more often a blend of all three (along with other factors).
Guess what? Dealing with hard problems is stressful. Easy problems aren't stressful (although boredom is for many), but if you wanted easy you chose the wrong profession.
So buckle up. Because if you cannot handle an hour sitting down and working either with an interviewer or under their observation/guidance on a problem, how are you going to handle it on the job when it all kicks off (whatever "it all" happens to be on the occasion in question)?[1]
[1] This presupposes the interviewer isn't an asshole, and can be depended upon to behave like a grown-up. I have borne direct witness to, or heard report of, far too many interviews where the interviewer appears to be on a mission to prove they're smarter than the candidate. If you are unlucky enough to be interviewed by somebody who behaves like this you may be better off looking for a position elsewhere even if you're offered the job.
For those looking for actual data, like the post’s author who says as much, one could turn to the classical music “industry”. This audition process where “the best man wins” was exactly that for many, many years, until someone thought to put up a screen and down a carpeted walkway, and well, what do you know?...when the candidates were truly anonymous, women started winning. Here’s an article http://gap.hks.harvard.edu/orchestrating-impartiality-impact..., one of many on this study. So that trick was revealed, but ultimately, the idea that they would select the “best” players of orchestral excerpts resulted in them looking for players who rivaled computers in accuracy. Unfortunately, that isn’t really what one wants in an art. Anyone there has the accuracy required in spades. Accuracy beyond that is just meat compiling. It turns out that classical orchestras populated by this kind of person lack morale, creativity and vitality and don’t evolve with the culture, so these organizations have become culturally dead and are going bankrupt all over the country—this after their members were forced to let go of any real compensation and all dignity. The problem is that most organizations like these and tech companies are top-down. They are run like the military; their basic training programs are even called “boot camps”, and like boot camps, they won’t get you much. We need to take a look at the whole system, I’m afraid, or it will suffer a fate similar to music in this country, because that’s what happens here, eventually.
That's just the way it is.
In the gig economy it's important to be able to show to your client how the final product looks like. I.e. a portfolio. They have money just for one run, one try. Risk aversion is high.
Companies that truly want the best and have a process to filter for the best are exceptions. Maybe less than one in a thousand.