How to hire programmers using online coding test
recruiterguide.hackerearth.com
recruiterguide.hackerearth.com
Before hiring, tech fit decisions are made by elaborate pretend tests and painfully detailed technical discussion upon people who supposedly needed to gain years of experience in a specific field. After hiring, tech fit decisions are made by a PHB glancing at the cover of PC magazine and saying "make it so, number one" and the existing staff starts googling furiously and supplies something workable in a week or two even if they never heard of the technology the day before.
At 2pm "Work at home, are you crazy? Without my direct supervision hovering over your shoulder I'd appear unnecessary and you can't be trusted to accomplish anything at home and nothing can ever be accomplished individually, things can only be done by a team where a sociopath will take all credit for the teams effort" or, uh, usually a little more carefully phrased but this is what they actually mean. But at 2am the tone changes "well of course you can work at home at 2am to fix any emergency all by yourself in a completely undocumented system you've never heard of before and we trust you'll always be available 24x365 and never out of town or drunk"
The approach taken in the article is not explaining how to HIRE programmers.
It is about how to ELEMINATE them.
// Your code
if "Time is a valuable resource."
Don't waste employers' time
end
// Adds candidates' time to the case
// Decreases listOfCandidates() loading time.
if "Time is a valuable resource"
Publish a well-detailed job position
endSurely there must be an algorithm for that?
If they pass that, we don't give them any more programming tests. We give them a hands-on sysadmin assault course problem, where they have to SSH into an EC2 box that has been carefully broken by our devious chief sysadmin, find out what's wrong, and fix it. They do it in a tmux session, with some kind of audio link (Skype, Hangout, etc), so one of our guys can watch and talk to them as they do it. I've listened to several people running through it, and it sounds terrifying.
And BOOM, suddenly you find out that the large instance on AWS, cannot handle the traffic, you are befuddled its not like the server is getting million hits/sec. You thought a large server would be enough to handle a single wordpress blog, and you realize that's not the case.
Now you go back and start fixing your self hosted wordpress blog, you feel sad that you missed out on audience because the server crashed, your work gets derailed because now you also have to maintain the blog server.
I agreed I too would have been irritated to see the link not working, but its not that you switch to an x-large instance each time you post a link on HN. Sometimes you just want somethings to work. But alas that's not the case.
I'll use myself as an example: I'm a self-taught developer who has built and managed high-profile, high-usage websites/web applications. I probably could not pass one of your algorithm-based FizzBuzz tests, but if I was hiring and you suggested that a site should scale simply because you threw it on a large AWS instance, I would not consider your candidacy further regardless of your ability to pass a FizzBuzz test.
You want to focus on the product, fix bugs, look after feature request, produce high quality content. And then your blog crashes and things get derailed.
That's precisely what you said:
"You thought a large server would be enough to handle a single wordpress blog..."
> when you are swamped with so many things you don't want to spend too much time on things like maintaining your blog.
A competent developer should be able to set up a load balanced WordPress blog (with Nginx/PHP-FPM, Varnish and a separate database server) in a day or less. There are a number of well-written tutorials on this.
As a candidate, I understand the system is incredibly broken. Maybe one way out is to have recruiters who have a technical background. I have seen that happen with exactly one company where the lead recruiter had a BS from MIT. She was awesome and a genuine pleasure to talk to. Also, she had enough of a understanding of the company stack that the interview acted like a semi-filter.
I'm in the process of hiring again as the team is growing and this has struck a chord with me.
I tend to see a resume, if it looks at all decent I will do a interview. Some would say do a call or give them a test to weed out etc. I've done enough interviewing to know that
1) people almost always are different than what their resume portrays for better or worse. 2) body language is 90% of our communication so why waste time on the phone. 3) If its so important to get someone I deem as a good fit I want to meet them. 4) I "waste" a lot of time interviewing, but I get better at it each time and I've found people who would otherwise never have gotten into see me that were spectacular hires. Also I have lots of friends who hire and sometimes someone is great but not for me, the favor has paid off when its returned.
The bottom line is people are unique in many ways. Why try to push them into a standard mold or discard them if they don't fit.
P.S
With regard to testing I usually tailor a test of 5-6 questions AFTER the interview so I can evaluate anything I am not sure of after speaking to the potential hire. I also let them do it at home with their tools. I don't care to see if someone can work with me hovering over them, because I have not intention of doing so on a day-to-day basis.
http://webcache.googleusercontent.com/search?q=cache:http://...
I interview A LOT. Almost 100% of the candidates I receive cover the basic programming abilities. The real problem is that they don't know how to THINK.
I talk with them 20 min on the phone when they have to present a problem they had and how they solved it. Then I try to find out what they find interesting.
Giving a mini project is a good idea, but it does not scale well, and still requires a lot of resources top candidates will just not be willing to invest.
If there were 20 plumbers and only 10 plumber jobs, I could get away with jerking them around forcing them to do crossword puzzles before hiring them to fix my toilet. That doesn't mean the selection pressure would magically result in the guy most likely to fix my leaky toilet; much more likely it would result in the guy best at doing crossword puzzles. Unfortunately I now have a concrete deliverable of a leaky toilet to fix, and the best crossword puzzle solver in the market trying to fix it. That isn't going to end well. Especially when the best leaky toilet fixer in the world knows he can refuse the crossword puzzle test and get a job via his rep or connections so he never even applies. So its not just a random selection of applicants, but a random selection of an inferior population.
Don't get me wrong, it would be possible to pick nearly worthless questions which prove nothing about a candidate but a well designed problem requiring a few dozen lines of code and no 'trick' or specific knowledge can tell you more about an applicant than many hours of trying to code on a white-board or doing silly logic puzzles.
The article suggests (1) using a simple but relevant exercise and (2) using it as a screen, to select people for a real interview.
The analogy with plumbers would be something like asking them to list the tools they would need to replace a hot water cylinder. For someone who knows their stuff, it's trivial. But it will weed out the guys who are bullshitting you. The guys who can give a good answer, you ask to come in for a proper interview.
2. ?
3. Profit!
Interviews are a two-way street. If you're looking to hire an experienced developer and the best you can do is ask him to do something that he'll probably never have reason to do on the job, you're sending a pretty strong message about your company.
If you're going to ask somebody to code (or solve a problem) as part of the interview process, there's nothing worse than being lazy in creating the task. "Print prime numbers below 100" falls under the lazy category.
Happily, that is not what the article is suggesting, nor what anyone actually does. The article gives three steps: first, a trivial programming problem, second, a challenging programming problem (here delivered as homework), and thirdly, a face-to-face interview to check cultural fit.
The purpose of the trivial problem, as explained in the article, and explained by anyone else who has ever advocated this approach, is simply to weed out people who just cannot code at all, and to do that early and cheaply.
My current employer uses a trivial programming problem like this as an early screen (it comes after our recruitment guy has read their CV, but before we put them on the phone with a developer). Therefore, i, and many of my colleagues, are experienced developers who have been given a trivial programming problem to solve. It did not any of us a message which put us off the company. I have not heard of anyone being asked to do this test and losing interest in working for us as a consequence. On the contrary, i was pleased when i was given the problem as an initial screen, because it told me that the company was serious about hiring people who can actually code, which puts them head and shoulders above many employers!
I see lots of references to "culture" in hiring comments, so I'm not trying to call you specifically out, but what does it mean? It always bristles me, like if your criteria is "white men only need apply" well just outright say it, don't tiptoe around it. As a suggestion to anyone on HN who does hiring, if for example you're not going to hire my wife because of her ethnicity or because she's a woman, just put it in the job ad and save everyone involved a lot of time and tiptoeing around "culture".
Its somewhat frustrating that "what defines a good developer" to a boss might be being a white male brogrammer from Stanford. But whats even worse is the uncertainty of not understanding what the criteria even is. Is it only whites need apply, or only men need apply, or both, or neither? That is what makes it even more aggravating. Its not 1950 anymore boys, its a multicultural world, so demanding a certain culture/demographic fit is going to result in epic fail if any competitor is smart enough to (gasp!) hire women or people of color or dare I suggest it, non Stanford grads.
Startups and people who hire should realize that they're turning off good applicants when they start talking about "culture" because people of color are going to assume you mean white people only, women are going to assume you mean hire men only, etc. As a business decision its just a bad move.
In fact, the louder and more explicit they are about their dumbass culture the better, because it makes it easier for me to avoid wasting my time with them.
In no case is the person wrong in what they want, but it might not fit in with the current culture, which can be bad for everyone involved.
That said - I agree that this can manifest itself as a never never land (the culture is drink till 4am on weekdays!), but I do still think that it is an important aspect of any organization, and properly applied can be beneficial.
Culture is not issued at your previous employer along with the company policies handbook, id card, and a dress code.
Culture is what you are, not how your immediately previous employer temporarily defined you while at work. My culture is I'm a northern European descended male living in USA, for my whole life I've been whats now called a "maker" for decades before being a maker was cool, religiously unaligned with any particular deity and live and let live with everyone else's imaginary man in the sky (and why is it always a "man in the sky" anyway?), educated in the classical "great books of western civ" sense, and my tastes in entertainment are vaguely European in outlook for example sex is great fun but violence is sick (I'm not a big fan of hollywood gorefest movies). And being a mostly German descended guy I have a pretty tidy front lawn in front of my house, that stereotype is often true.
Culture is not "$current_employer tolerates flextime so I could never work in the future anywhere else that doesn't have flextime because all human beings are permanently inflexible, oh wait, actually I've never worked the same schedule at two difference consecutive jobs so I guess it doesn't matter anyway"
Everyone knows 60 hour weeks means reduced productivity compared to 40 hour weeks, so I can slack off if I have to, to fit in with the other burned out sleep deprived zombies, but rest assured I'm only going to work like a demon for 40 hours and then surf the net for 20 because you get there the fastest by going slow.
For us, it just means "will we be productive and happy spending the working day with this person?". Somebody could be a great programmer, but might not be someone you're going to get on with. They could be too quiet, too loud, too aggressive, too passive, too arrogant, too sensitive, too confident, too insecure, strangely obsessed with tubs of jelly, etc. Great programming skill is a great resource, but if a programmer can't get on with their colleagues, that resource cannot be exploited.
We would not rule you or your wife out on grounds of gender, ethnicity, nationality, religion, university, taste in literature, or the state of your lawn. If you for some reason wanted a job with us, and passed our technical tests, then all we would want to know would be that you could sit down and be an effective pair in programming, and stand up and be an effective participant in a discussion. Ideally, we would also like you to be good company at lunch or in the pub, but that's not essential.
How about too picky? Hint hint.
> We would not rule you or your wife out on grounds of gender, ethnicity, nationality, religion, university, taste in literature, or the state of your lawn. If you for some reason wanted a job with us...
How generous of you!
The language one chooses to use can reveal a lot about one's thinking and here, "rule you or your wife out" and "for some reason" is, I would argue, very telling.
I know others will probably disagree, but I feel this way about brain-teaser questions. I remember one very specifically from a ex-Microsoft employee who interviewed me: "If you're alone on a deserted island and you have two unmarked jugs, one that can hold 3L of water and the other 5L of water, how would you make sure you had exactly had 4L of water?"
To his credit, he helped me through the question and I was able to figure it out.. but the first minute was spent trying not to be a smartass and ask "Is this a problem that programmers in this company frequently have?" (I did get the job fwiw)
If you started life as a chemist this is kind of a lab joke as in its funny but if anyone in the lab tried something this wasteful of time and product or risky of contamination if you reuse the product, they're too stupid/lazy to work in your lab. The correct answer is supposed to be walk to the stockroom and ask for a 4L container.
Its about as useful as a practical programming test as asking who won the 1984 baseball world series, in other words a complete waste of time.
Fizzbuzz at least is an excellent test of the modulus operator (oh boy is that ever useful to me on a daily basis LOL) and a so so test of control block and conditional knowledge which is actually useful on a daily basis.
I used to do a large amount of phone screening for Amazon, probably over a hundred in all by the time I got sick of being a professional interviewer. I experienced much of what the author did: the bulk of the candidates had zero algorithms or data structures knowledge and could not program. Not "can't program well" or "can't program at a professional level", we're talking "I don't trust you with a simple shell script".
Now I work for a rather more obscure startup - we're relatively well known in the tech scene around here, but we're by a very long shot not a mainstream household name. The caliber of people that come through the door now is vastly different. I've interviewed some duds here, but have never interviewed someone in this position that just outright couldn't code.
The question I have for the author is who he hires for, and how much they pay. The low end of the software industry is plagued with people who have no business calling themselves programmers. It's really a vicious cycle - there's a large segment of our industry where the typical skill level is disastrously low, which causes ever more bureaucratic and ever more absurd filtering mechanisms to be invented, which further alienates programmers who have any real choice in employers.
Nowadays I'd be mildly insulted if given a FizzBuzz during an interview - but I used to do the same out of necessity, because 25% of my candidates couldn't pass it.
If would be nice if the recruiter agencies (who are actually getting paid to do a job), would hire people qualified to understand if a candiadate is or not up to par for the job from a quick email or phone call.
I would never hire anyone without having an actual conversation with them. Whether I can get along with the person, if they will be a good fit for our team is the most important criteria.
Test taking is not an important skill for any job I'd be hiring for.
Case in point, the article does not recommend hiring without talking to the candidate.
Now that it's recovered I see that it was just to shill the author's particular coding test.
Yawn.