P(we hire you) =~ 0.0016
davidlynch.org
davidlynch.org
Turning down a lot of candidates is not the same as finding good candidates. If you are turning down that many applicants, all it means is that your applicant pool is almost entirely bad candidates. That merely implies you are failing where it matters most, getting awesome candidates interested and excited about working for you.
I bet you could drive the P down even more. You could write worse job descriptions that get even more of the wrong people to apply. You could post the job listing on job boards like Monster or Dice, where there is a huge pool of un-hirable candidates who will apply to 20-30 jobs per day. I bet you could make it so bad you will have to hire someone to be the initial filter of candidates. Congratulations! You now have an HR department. They will arbitrarily filter candidates before the people actually competent to filter candidates get to see them, based entirely on buzzword bingo and 'number of years of experience'.
You are failing and you're proud of it!
Part of the problem is people don't realize that they are doing it poorly.
You would think the 0.16% success rate would be a pretty big indicator.
Based on my own experience, there were a number of points in the article that I have found to be not effective ways to hire programmers. And that is fine. They are free to hire people as they wish. But placing all of the blame on 99% of programmers being unqualified seems to miss the mark. It is more likely the hiring process causes them to miss out on the really good programmers.
"It turns out to be quite difficult to hire good developers. I’m involved in the hiring process at deviantART, and it has opened my eyes to just how unqualified the majority of applicants to these jobs are."
There seems to be a complete lack of connecting the dots from "this is unusually difficult" to "what can be done to make this better/easier/more efficient." And given that this failure is centered around software developer positions... it doesn't say good things about the kinds of problem-solving that goes on in their culture.
1. Do the phone screen first. Frankly, in my experience, you can filter out a lot of people in the first minute; and
2. Then do the test.
Speaking of the test, I had a quick look and it takes time. How much time? I'm not sure as I didn't look at the starting code, just the list of tasks. Maybe it only takes 30 minutes. Maybe it might take hours. I really don't know.
Either way, these sorts of things are huge barriers to entry because, as someone looking for a job, I also have to decide on how much time to spend on you. The fact that this test is just to get to the phone screen lowers the amount of effort I'm willing to put in.
Also, you don't need terribly complicated coding problems to weed out early stage candidates. Honestly, getting someone to code a relatively efficient (at worst O(n) space and time) factorial function will tell you ~95% of what you need to know from this test if you can witness them actually doing it. Seriously. So screenshare with them and get them to do it. No editor. Just a Google doc or similar.
That all being said, I'm really shocked how woefully unqualified an awful lot of programmers are at... programming. The sort of acceptance numbers talked about here and at places like Google (disclaimer: I work for Google) are more testament to the number of clueless cowboys infect our industry than anything else.
Also, bear in mind that an interview is a two-way street. The more effort you expect from the candidate the more they need to want to work for you before they even start interviewing. Google, Facebook, Twitter and the like have this kind of developer gravitas. Do you? I don't know.
Lastly, the three month trial thing is, for me, a huge problem. After all this there's a 20% chance I might lose my job for no reason other than you may not like me?
One reason Google is so selective is we take employment seriously. Leaks and criminal offences notwithstanding, it's actually pretty hard to get fired from Google and those decisions are not made lightly. That may have some negative side effects but overall I think it's a good thing that engineers aren't constantly worried about being fired.
Vague requirements like "add support for our new data format" mean nothing to me in this context. Am I supposed to add error checking? Should I automatically detect which format is used? I have no idea how much effort is expected, and so I'm not even going to try because some less talented person may have had more time to work on it than me.
The great thing about FizzBuzz is that it's the kind of thing a decent programmer can knock out in a minute or two (so they're not wasting too much time when they speculatively apply) and yet it's just sufficiently difficult that people who can't program at all are easy to screen out.
The Deviant Art test however is something I would consider a dull day of work. If you want me to prove my worth by challenging me to an interesting combinatorial optimization problem, I'll bite. However if you want me to prove my worth by testing my ability to perform the kind of task that usually makes programmers think they should be looking for more challenging work, then good luck to you.
1. http://www.itasoftware.com/careers/work-at-ita/hiring-puzzle...
I think the reason for the low P(we hire you) is that you may be accidentally skimming off the cream here. At the very least try getting people on the phone first so they can say "hey these DA guys are pretty cool and interested in me, it's worth jumping through a few hoops", or make the test seem interesting/fun/challenging
If you can do that, you'll win a Nobel prize. (Yes, I know there isn't a Nobel prize in computing: You'd win a Nobel prize in physics, for showing how to store Θ(n log n) bits in O(n) space.)
Heck, if you can compute n! in O(n (log n)^(2-x)) time for any x > 0 you'll probably win the Turing award. Such an algorithm would be really really big news.
The space usage for this function is obviously constant, and the number of multiplications will be linearly dependent on the number.
If you mean to refer to the 'factorial function' as a mathematical object, and point out that bignum operations have different complexity... well that's true, but irrelevant to OP's post, and there are less snide ways of saying it.
If cletus meant "factorial as long as the result is a 32-bit integer", I don't know why he would accept an O(n) solution to that: I wouldn't hire anyone who didn't immediately produce the obvious O(1) algorithm using a 13-element lookup table. (If he meant "factorial modulo 2^32", a 34-element lookup table suffices.)
As for being snide... yes, but I'm tired of the myth that Google is full of (and only hires) algorithms gurus. I have yet to see any indication that Google requires any more than a minimal undergraduate level of algorithmic competency.
Because it's a question used to weed out early stage candidates. It's a fizzbuzz test. The point is not to see if the candidate knows the algorithmic complexity of arbitrary-sized integer operations, and neither is it to see if he knows whether the factorial function grows quickly enough to make a look-up table worthwhile. It's just to take a very simple function and get him to write some code in a text editor.
Or at least, that's what I read from the OP's post. Perhaps he meant otherwise.
But it is not a factorial function. It happens to have the same result as the factorial function as long as the result is less than what can be stored in a fixnum int in C.
But it is a very real possibility that that isn't enough. I always wanted to find out what the sum of the factorial of the first 100 numbers was but what you claim is a factorial function can't do that.
Ultimately "because we don't like you" is the reason you'll lose any job. The main thing that makes us decide someone isn't working out is if they're not performing, though. I don't think I've seen anyone get dropped after the trial for any other reason.
Yup. Your hiring practice better be up to the task of bringing in the right people at the outset, because the top candidates aren't going to tolerate a trial period. Not in this market for their talents.
I also think the trial approach is potentially harmful to the company. There's a reasonable chance that the most ambitious and positively disruptive people are going to clash with the existing team at the outset since they're not part of the groupthink.
Sure, if a guy turns out to be a mistake you can and should fire them, but starting everyone on probation is no good.
There are employees who place a very high value on job stability. In the general case, these people are more 'loyal' in that they are less likely to jump for better pay or what have you a few years down the road.
But depending on how you run your company, it may be rational to select for the sort of person who does not expect job stability. I'm in this second class, personally. Yeah, I'll jump if I get an offer that is 20% better, but on the other hand, I maintain 'how to fire me' documentation, and I expect that if I go through a long productivity slump, that you will fire me, and I won't hold it against you. I expect and try to give absolute honesty, but as far as I am concerned, loyalty is a lie. (I know that for other people, loyalty is important and not a lie. But if I claimed loyalty, I mean, beyond honesty, I would be lying, and if a company offers me loyalty, I am, ah, suspicious.)
I'm not saying one approach is better than the other... Both approaches are valid, and they both get you different sorts of employees. In fact, the optimal solution may be to use a different approach for different types of employees; this is traditionally done by having full-time employees and contractors (now, the way most companies hire contractors is hugely inefficient; the body shop eats so much value that this is probably a bad idea. But if you could have some other way of setting the loyalty vs. no loyalty expectation, a company could benefit having both family and mercenaries. We are good at different things.)
> Generally speaking, there are two approaches to relating to other people in the business world. The first approach is transactional and legalistic: work is primarily an exchange of labor for money, and agreements are made via contracts. Enforcement is provided by organizations, especially the legal system. The second approach relies on trust, verbal agreements, reputation and norms, and looks to the community to provide enforcement when necessary.
Now I'll certainly agree many employers pay lip service to "loyalty" as a means of controlling costs and turnover without it being in the least bit reciprocal.
So while this may not be your typical experience (it certainly isn't mine) I can tell you from experience that Google takes this pretty seriously.
That all being said, you are basically a fair weather friend and, to be perfectly honest, you'll never see the rewards that only come from sticking through the hard times, which is actually your loss (IMHO).
Can you really be "Friends" with an entity who has a legal responsibility to "increase shareholder value?" I mean, sure, I have a relationship with the corporation, and there is trust involved (well, mostly they are trusting me. I'm just trusting them to deliver up my paycheck, and while they are usually late, I almost always get paid in the end, so they are trusting me a lot more than I am trusting them.) but yeah, I don't think I can say I'm friends with a legal fiction.
I think that if google falls upon hard times, you will see that they are also "fair weather friends" - It's possible that google employees, after being shown such kindness and loyalty during the good times, will feel very betrayed.
And rewards? What rewards? It's very rare that employees get substantial rewards when the business does well. It happened with google, yes, but that's the exception, not the rule.
But then, really, I don't actually want to be an employee. Right now, I'm not. (I mean, technically I'm an employee of prgmr.com, but I'm also the majority shareholder, so I don't think that counts.) But, sometimes I need more money than I can earn on my own (I'm not a very good business person) and in those cases, I usually look for short-term work.
The thing is, short-term work is pretty miserable for people who actually want to be part of the family (or, as you said, the standing army. An apt description; The thing that scares me the most about enlisting would be the fact that you can't quit.) They feel like they are being used and abused. And sometimes, companies need short-term work. It's hard, really, to go tell someone who is the type that expects loyalty "Hey, you- help us evaluate these outsourcing companies that will do your job after we fire you." or "go write an automation system that will mean we can fire 4/5ths of the people on your team" But a mercenary? sure, why not. I want to be here forever even less than you want me to be here forever.
When I do work for other people, the best case, I think, both for me and the employer is if the employer needs someone short-term; One of my more recent consulting gigs, the client was trying to outsource their linux SysAdmin stuff, and they were kissing a lot of frogs. They needed someone in-house to help meanwhile, but it was clear they were going to drop that position as soon as they found the right foreign contracting company. It was win-win for me, 'cause I wanted some cash to spend on my own projects, and I didn't harbor bad feelings towards the people who they were trying to get to replace me. The job lasted longer than expected (oh man, so many frogs!) but they eventually found a company out of mexico that was pretty good, and told me thanks.
I mean, there is certainly room for both in the marketplace. for some roles, you really want someone who will stick with your company through thick and thin. But those same people generally do badly in roles like the above where you are essentially asking someone to help you fire them, or where it's glaringly obvious that the job will end shortly.
>1]: http://cdixon.org/2009/10/23/twelve-months-notice/
Personally, I disagree with Cdixon there; I think a loyal employee is far more likely to sue an employer if things go south than a mercenary. I just want to get paid; There's rarely more than a months worth of compensation on the table, and that's not a lot of lawyer time. By becoming a contractor, we give up almost all the rights normal employees have. Contractors, especially direct contractors are often in a better position to cause legal trouble than employees; the IRS is very liberal in what it defines as an employee, and very conservative in what it defines as a contractor. As a contractor, you don't threaten to sue just like you don't threaten to bring a gun to work and start shooting.
But employees? It seems like one out of five employees who expected loyalty seem to think they have some sort of discrimination lawsuit even when they get the sweet, sweet first round of layoffs; the one with the nice severance packages.
Mercenaries also operate on trust. How do you think I'd get my next gig if I sued the last guy? Trust and reputation are essential to getting new gigs. If you want to be literal about the mercenary bit, you aren't going to hire me if you think that I'm going to come back and sack your city after I've done the bit you've hired me for.
I'd argue that a reputation for being litigious is almost a 'never work again' thing for a contractor, and we know it, while most employees seem to think they have "rights" they can assert without consequence.
And really, helping people after the layoffs, I think, is largely a "how do you want to be perceived" thing; I know once I was stepping down my own company and had to get rid of someone. I ended up switching jobs myself, and recommending my employee to the old employer. He got the job (and a massive raise over what I paid him) and the old employer was really happy. They gave me several other gigs down the line, and the guy who worked for me has recommended other employees to me since then. Trust and honesty are important on both sides, and when you do need to fire someone, if you can soften the blow, do so. At smaller companies, it really does make sense to help employees get other jobs; it usually makes the employee feel better than severance, it's usually cheaper than severance, and you get goodwill from the new employer on top of all that.
At larger companies, it seems, giving people some cash and maybe some retraining is probably the best you are going to do, but that's still important. The people who continue working there don't magically stop talking to the people you fired after the layoffs; and you are expecting the remaining people to work harder than they were before, so if you don't want your best people to jump ship, you want to make them think that they will be taken care of if they are caught in the next round. Of course, the deceptive bit of this is that usually by the second or third round, when you are letting your better people go, there isn't enough cash left to give good severance pay.
But yeah. I dono. It is possible that my attitude makes me a worse leader. A more honest leader, but maybe worse. I don't understand it when people give me loyalty. My employee has worked for me far longer than I expected. He is currently interviewing at google, and he tells me he feels like he is betraying me. (google approached him, not the other way around.) From my point of view, this is just the way things work. I hire less experienced people, give them some training, and then as they gain experience, they move on to bigger and better things. At this point, the guy is better than I am in a few areas; I would have expected him to leave more than a year ago. Getting mad about someone leaving me for a substantial raise would be about as rational as getting mad at the rain for making my roof leak. Rain happens; If the roof leaks, that's on me.
Looking at your coding exercise, it's a small set of code, but it would be more than an hour or two for me (or presumably anyone) to do all of the improvements, find all the bugs, fix them, do the extra credit, make sure it's not sloppy before I turn it in, etc. So way more commitment than just sending in a resume or writing you an email.
Reading your blog post, you guys seem to want rock star technical candidates, but with that job description, you're going to attract people who have done a few PHP web sites, maybe customized a Drupal or or Joomla or Wordpress site.
When I've interviewed and hired developers, our best ones always came from friends or recommendations, never from job postings or job boards.
Google doesn't do it, Amazon doesn't do it, so if that's who you're competing against for developers, why would you do it?
Also, if you can't be bothered to spend 30 minutes proving that you can code, how much do you really want to work there? The only time I've done one of these challenges was for a company that I was really excited about working for.
Contrary to latch's comment here: http://news.ycombinator.com/item?id=3062001, Google does tweak the process from time to time.
The hiring process isn't one sided, especially with the demand for talented developers at the moment. So not just how much I want to work for the company but how much the company wants me to work there.
1. Another programming problem. Okay, if it doesn't take too long, I'd do it.
2. 60 day fuse. No way. I've worked under a Sword-of-Damoclese clause before, and I won't do it again. This is a deal-breaker, and it's probably sending a very bad message to a large group of good engineers, and you're missing out.
Fire for cause, or whatever, if you have to. But I won't work in an environment like that, and (I guess you'll just have to take it on faith) it's your loss, not mine.
So, your strategy is to deliberately make your job offer sound as unappealing as possible in order to find the people-who-call-themselves-programmers so unemployable and desperate for work they'll subject themselves to a trial period? Well, I would say that strategy, in terms of achieving its goal of attracting the desperate, is working out wonderfully for you all.
From my own experience watching as my manager tries to find good people and from everything I've read, it IS hard to hire good developers. But it seems like you guys are making it much harder on yourselves.
I have screwed up before. It sucks. It is painful and horrible to deal with someone who is not working out. But I haven't blown a "hire" vote in 18 years. Hundreds of interviewees, dozens of "hire" recommendations, zero bozos.
It sounds to me like you are insecure in your hiring practices, to the point where you tell people "Quit your job, move your family, turn your life inside out, whatever; but you're on probation here and you're gone if you blink funny."
Maybe it's just my past, where I was in this position (and it worked out really well, btw). But I wouldn't do that again, and I wouldn't do it /to/ anyone.
Good luck. I hope it works out. But I cannot imagine a worse way to enter a job.
1. I got as far as a promise of a second interview that never materialized. I found it annoying to be told that they wanted to give me another interview, then that they 'moved forward with another candidate,' then to see that the position is seemingly unfilled months later. (I'll note that they were posting in the "We're Hiring" threads within a week or two)
2. Between stages in the interview process I would be told that I would be hearing back from them 'shortly' or 'soon.' This ended up being an average of 3 weeks between stages in the process. Which was rather frustrating.
[edit] I'll at some other notes:
1. The programming problem doesn't have to be perfect. Mine wasn't. They pointed out that my code was susceptible to SQL injection during the first interview. And I also missed a JavaScript gotcha (that was obviously placed there intentionally). [Obviously you want to do as well as you can on this, just sayin' that they aren't requiring 100% on the problem to get your foot in the door.]
2. I was able to get an interview with little PHP experience.
(Throwaway account, as I would rather not publicize that I'm looking for a different job. If you're looking for the same type of person dA is looking for though, feel free to reply to my comment with contact info.)
You get 2 hours, a problem that doesn't take more than 30-60 mins to solve, and an internet connection in case you want to look something up.
I agree with other posters here that a very simple problems weeds out a large chunk of the inept folks.
We had people just sit in that room for two hours and all they did was remove some lines from the skeleton that we have them. Or people who just left this simple exercise and went home.
I disagree with others about doing it life via a shared session. Programming and thinking is a private affair (IMHO), there is some exploration component in it. Nobody will want to explore in front of a panel.
In the end we don't even look at the test unless there is disagreement between the interviewers, or we identified a specific concern about problem solving.
Will you be on probation after that grammatical "bug"? By the way, you guys sound pretentious as hell. I'm surprised you don't lose more than 20% in the three month trial.
What's the point in the trial? If it's not working out, just fire them.
I feel like this is indicative of one of three scenarios:
1) For whatever reason, there is a high turn over rate at your company. This isn't always a bad thing, but if a company is always hiring, it's usually because they are consistently losing people as well.
2) You are experiencing crazy explosive growth, so much so that every month you need to hire more of the exact same positions.
3) The requirements for the job are poorly defined. This would lead to a lot of people (mistakenly) applying to a job that they have no hope of filling the requirements for.
Whatever the case may be, I really think the only reason you guys are getting any kind of backlash is just because it is becoming a common theme to apply to DeviantArt and never receive at the very least a "Sorry, not what we're looking for". Do it to enough people, and there's bound to be a response.
EDIT: There is a 3 month period in which 20% of new people get fired?! No wonder you are always hiring. You are effectively telling people to leave whatever job/security that they have, then cutting a non-trivial amount of them loose because they weren't a good fit? That's not their fault, thats a problem with the hiring process. Who knows what it does to the moral of the other workers. Oh wait, they are all remote. No one ever finds out...
That puzzle you mention in your post, would be a great way to give people an example of the kind of problems they will face before they choose to apply.
3. we get very few off-topic applications. most who apply do have some level of experience to qualify for the position. yes, the first screen is subjective, and we clearly make mistakes if all we have to go on is the CV without a single link to one's work. Thats why the position description suggests submitting code samples like a link to the github account would be very beneficial.
we do respond with an automated message acknowledging the receipt of the application and it spells out what to expect. We contact again to request code sample and exercise of those who pass initial screen. unfortunately sometimes those emails get trapped in spam filters. We did used to send rejection messages in the past, but this also did not work very well because many applicants expected very personal response and coaching on what went wrong. As all of us are also full time developers, we just dont have bandwidth to do that. but based on your guys's comments we'll see how we can do it again.
Is this common?
Here's the direct link: http://www.codeeval.com/public_sc/47/
Give it a shot.
You see, the best candidates stay on the market for maybe a week. You'd be lucky to interview them once. Nobody who is truly good will jump through your hoops.
You're just hiring the best of the ones willing to subject themselves to this ridiculous process, pretty much some desperate developers who are barely good enough to pass your tests.
And a 3 month probation - no way in the world, I would only do it if you paid $200K + bonus. The market is desperate for good developers, I'd have to be completely desperate to be treated like that.
You're hiring... the ones willing to subject themselves to this
ridiculous process, pretty much some desperate developers