How We Hire Developers at Treehouse
wellbredgrapefruit.com
wellbredgrapefruit.com
As a developer I don't like the idea of having to do a 10-20 hour project as part of an interview, but at least they are compensated for it. I would love to hear thoughts from developers who have done a "project as an interview". Positive or negative experience?
I got both the jobs where I did the week-long projects. They took a while, but it was completely stress free, and allowed me to show what I can do under more realistic circumstances. I realize that some "rockstar" programmers might scoff at someone daring to take up so much of their free time, but these are the people who probably do well at the classical interview, so more power to them.
Overall, I enjoyed the project-as-interview process. Two things though: (1) I was a youngin at the time without much responsibility. Ask me to do the same thing now, with a family, and it can't be longer than a few hours spread over a week or two, or I just don't have the time. (2) As an interviewer and fellow human being, I'm very respectful of people's time. I'm uncomfortable asking someone to commit so much of it, even though for me personally it was OK.
One problem with step 3 is that it filters out engineers who are busy with their jobs and life, and cannot easily afford doing 20h extra work (even if it's paid work). Especially if compensation etc. are not negotiated beforehand.
Why do you want to work at Treehouse?
makes perfect sense to you. An applicant, however, didn't know your company 15 minutes ago, and his main motivation is either money or finding a job that fulfills his desires.
"Why do you want to dedicate the next few years of your life working on this instead of something else?" If the answer is just because it pays more or the company is in a better location I think most employers would want to keep looking for someone that has different motivations.
I've never found that to be a great idea. In all honesty working on a stack in one company is much the same as another asides from whether management is an ass or whether they're paying you decently - that kind of thing. A passion for the company's mission is just nonsense IME - sure, the person might in theory like the idea of better traffic management (or whatever) but at the end of the day that's not going to keep them warm inside in the same way being treated with respect and an extra £20k/pa is going to, or in the same way that having their kid go to a local school and their wife or husband working in the same area is going to.
You've got things that keep people bought into a company and passion... I'm not seeing it as a big part of it, especially considering so many people are going to lie about. People are too alienated from the product of their work for it to be meaningful.
What I have found reduces attrition is if you train people in house, if you treat them properly, if you pay them decently - that sort of thing. Which is basically what people are telling you when they say they're interested because of the money, they don't know enough about the company to say much else.
Passion for a role, now that's meaningful - but it's not really a company-specific thing, it's more a work-ethic thing.
If you're the type that needs personal validation such that you would knock a candidate for saying something like "I need money, you guys are paying in money", then you simply shouldn't ask the question. Of course, people love to think that they have a highly tuned bullshit detector, are immune to blatant flattery, etc. The problem is everyone thinks that. We are all susceptible to these things and we need to make sure our process is such that its not biased towards people who are particularly good at telling you what you want to hear.
People forget that an interview is a dialog between two parties. As a candidate I am not trying to "win" by getting an offer; I am trying to find out if this is a place I actually want to work.
I took Ramit Sethi's freelancing course awhile back and it really opened my eyes as to how to sell your services to people. Worth every penny.
| I prepare for this question directly by perusing
| the company's website and Wikipedia page, if they
| have one, along with doing a little basic research
| on the industry before I go into an interview.
| The real way to answer this is to start asking them
| questions about their business needs and have a short
| conversation about it.
Not as big a problem as you seem to think it is. You have to realize, 99% of people interviewing won't know practically anything about the company they're about to go to work for. Doing some basic legwork is a very cheap way to stand out.
> To me, turning around and answering with a question comes across like I'm trying to dodge or deflect the question.
Or that the question needs additional context to be adequately replied to.
My problems:
1. Makes demands of the candidate before selling the role (in fact, explicitly!)
I recommend you don't do this. Instead, make a point of going out of your way to sell your company and your role to your candidates before you do anything to screen them.
This serves three purposes: (1) sure, it makes it marginally more likely that you'll keep high-value candidates engaged who'd otherwise fall out of your pipeline at/after "stage 1"; (2) more importantly, it helps disarm the interview process, both by establishing that you, the employer are going to put the effort and initiative into keeping the process running and establishing that tone for the whole process, and, even more importantly, by making the candidate actually want the job so they'll perform better during the process; and, (3) it's both the right thing to do and a noticeable difference from most firms' terrible hiring processes, which is good for marketing.
2. Is extremely subjective
Right out of the gate you're asking the candidate questions you can't possibly be benchmarking. "Code you're proud of"? What if the code the candidate is most proud of is something simple that been super useful on lots of projects? What if the candidate cherry-picked it from their code to demonstrate the most complicated stuff they've worked with? How will you compare it to every other code sample you get across the lifetime of your company? You're doing that, right? Keeping track of how well your hiring process predicts performance?
2a. Culture fit
Beware culture fit questions. You state it outright later in your process when you say (paraphrased) "you can train technology but personality mismatches are impossible". That's a recipe for hiring the same 10 dudes who like roughly the same X360 games. I know that sounds hyperbolic, but look around you at other companies and tell me that isn't a real concern.
Two other problems with culture fit questions: (a) they mask irrational responses from interviewers, which happen all the time. Some of your best performers do badly on interviews, because they just don't interview well. Some of those people (ironically, it seems to be those people in particular) will be hard on candidates who have the same problem. Why are you setting the expectation with your team that they should ding candidates for intangible, irrational, or subjective concerns?
In neither (2) or (2a) am I arguing that you should only be hiring based on measured defect density or ability to correctly estimate how much time feature X would take† or typing speed. If there are specific personality traits you need to select for, that's fine: just select for them deliberately, and train your team how to select for them, and track how you're doing.
3. The tryout project
This is going to make me sound like a jerk for at least 2 reasons I can immediately think of, but, to sacrifice (a lot of) tact for (a little) clarity:
I think less of teams that propose short contracting projects to candidates to try them out, because I would never agree to interview on those terms.
Here are the problems I have with this approach: (a) it invites/provokes an argument during your hiring process about what a good daily rate is, (b) it asks the candidate to negotiate an hourly rate in a situation where their leverage is egregiously compromised, which is unethical, (c) job searches are full-time jobs by themselves, and your best candidates are almost invariably already employed, so how could it be reasonable to demand they take a temp job with you as part of that search, (d) it requires you to demand the candidate produce billable work and assign the resulting IP before the candidate is sure they're going to accept the offer they might get from you, (e) the tryout project is deceptively low-information.
On that last point: I guess I've hired a person or two in my career that ended up not working out almost immediately. But usually, the dealbreaker problems I've had with coworkers or employees took months to surface. Lots of people can be productive for a short, high-stakes sprint. But virtually no dev team is hiring for that; they're hiring for the ability to be reliably productive, to exercise good judgement in design and estimation, and be compatible with the decision making process the team uses.
Yes, the tryout project proves the candidate can produce code you might deploy on your website. I hope I'm not the first to tell you that that is a very low bar; lots of high school students can do the same.
Please please please take this comment in the spirit I intended it. I expect lots of smart people will disagree with some or all of it, and would be happy to learn from them. I'm pretty convinced though that the conventional interview system that some of the smartest teams use is fatally flawed in a bunch of ways, and this hiring process is an opportunity for me to start talking about why.
%. One last point because I am on a tear about this lately
I would like to start establishing a meme: Job Interviews Are Hostile For Candidates.
Normal people don't enjoy interviewing for jobs. Normal people find the experience off-putting. They're nervous. They're dealing with a procession of people each of whose stated position is to judge them. Normal people don't like being judged by strangers. They sit there and read tea leaves from behavioral cues about whether they're coming across well. In my experience, this happens even in phone interviews, which (can we also establish this meme?) are the worst.
I am not saying don't interview people. I'm just saying when you do it, assume all the signals and readings you're getting from your candidate are janky as hell, because they are. Lots of strong performers can't do their best thinking and judging when they're conflicted and anxious. Compensate.
Whoah, long comment.
† BTW: my favorite dev interview question
†† There is a better way to write this paragraph but I'm pretty depleted right now so please excuse the clunky, combatitive tone.
IMHO any sort of interview process is a crappy way to hire people. The best jobs I've had, I got by meeting people informally. Jobs where I've gone through a really formal interview process have always turned out "just okay" at best.
I am not saying don't interview people. I'm just saying when you do it, assume all the signals and readings you're getting from your candidate are janky as hell, because they are. Lots of strong performers can't do their best thinking and judging when they're conflicted and anxious. Compensate.
It hits me straight in the heart. I get very, very nervous at interviews. For no fucking reason whatsoever. My mind goes blank. Last one I did (more than a year ago), the person told me that I was obviously not prepared to work under pressure. But I can. Hell, I own and run 4 businesses. Pressure is what's for breakfast. The issue is that interviews are focused on trying to get me to fail, rather than letting me impress you. Which is the reason I am so successful as a freelancer. As an independent, all I need to do is show you my portfolio, talk about your needs, and get on to coding. There is no interview. There is, however, a trial period where I show you what I'm capable of.
Its just strange. I'm building a search engine, and have built 3 lead generation applications. Yet for some fucking reason, I flunk interviews, and people won't hire me to write coldfusion. Still, can't complain. I make 5 times as much. Without having to deal with a dumb manager.
- I have been rejected in interviews for the most arbitrary of things. I was once rejected because I didn't know what __call__() did in Python, and because I didn't know about Partial objects in the functools library (part of the Python standard library). These are things that I was able to look up in < 5 minutes after the interview. This to me comes across as, "I don't know how to interview people, so I'm just going to ask a bunch of obscure 'gotchas' and hire the people that get them right." No one in their right mind is going to go into that level of detail when talking to a freelancer.
- "Having a Github account," seems like a requirement these days, but I don't think that anyone at any company I have interviewed with has actually looked at the code that I have written beyond maybe counting the number of repos and the types of projects. This seems like exactly the kind of thing that people would want to do to hire a freelancer, look at their body of work.
- It's presumably more difficult (more hoops, more paper work, more laws to run afoul of, etc) when terminating an employee, vs. terminating a contract with a freelancer. So there's less of an idea that you need to make extra sure that you're getting the right person.
- Freelancing is more of a B2B transaction than a job interview is. This seemingly puts you on more equal footing with the prospective client, than when you're running the obstacle course to 'prove' yourself for interviewers.
Theres only a couple of things that actually matter when you're hiring: a) do you trust the guy; b) do you think he can do the job; c) how hard will he work. Unfortunately a) is based on feeling, c) is nearly impossible to gauge but you can probably workout b) from a decent technical interview. Using anything else as a qualifier will open you up to deceit and really doesn't have a lot of place in your trade of skills for money, unless you're trying to create the type of company where people don't go home, in that case hire them young and dumb.
For the record, I have a personality but like 90% of the adult population, I'm a fairly decent person, can talk to pretty much anyone and can easily find common ground with most people. If you have a company culture that honestly forbids that, you've probably accidently hired a few asshats and that's probably something you want to kill rather than grow.
Wish the OP all the best though. :)
The example in the post actually sounds quite relaxed (environment of candidate's choosing) compared to what I went through in a recent interview for something akin to a data analyst position.
I was given an Excel sheet with a mass of sales data in it and asked to come up with whatever 'insights' I could in 15 minutes. On a Spanish version of Windows/Office, running on a laptop with a Spanish localised keyboard layout (which meant none of the symbols were where I would expect them to be on a US or UK keyboard). I must've spent about 5ish minutes of the allotted time just figuring out how to get the required symbols to show up :(
Sorry, but there's no way I'm going to do a 25 hour project for your company in the context of an interview, because I'm wasting time that I could use to interview for another 10 companies - which will most likely give a much larger likelihood of a job.
If you give exact requirements and give promise of a job if those requirements are met, I would consider doing the project. Anyone who does otherwise is foolish and objectively desperate.
Something I really liked about it was that they were fine with me searching up things I did not know (IMO, Googling is a skill and asking candidates to answer trivia questions solely from memory is quite unreasonable).
What was a bit inconsiderate was that they made me do this on a Mac. Now I understand most people have used a Mac before, and I certainly have worked with people who used Macs. Unfortunately though, prior to this, I had never actually used a Mac (let alone write code on it). I thought this could have been done better, but I do realize being someone who has never had the opportunity to use a Mac, I am part of an extremely small demographic.
I've read http://matasano.com/careers/ a couple of times (because perhaps sometime I'll be interested in applying, or perhaps because I may end up stealing bits of it) and I think you guys are on the right track. I'd be interested to read about your experiences refining that process and what sorts of things you've learned along the way.
Culture fit seems to be one that leaves a lot of wiggle room to assemble a team of clones, but you really have to look at what that company stands for. I'm more inclined to want to interview for a company that has taken time to define it's culture, and what they stand for, and aren't just looking for technical skills alone.
What's not fine is a tacit understanding that candidates should be hired or negged based on arbitrary attributes, gut feel, or "would I enjoy having a beer with this guy".
Like Mechanical Fish said: you have to hire like you mean it. That means, if there are nontechnical attributes you're screening for, you need to define them ahead of time and deliberately design a screening process that selects for them.
I think less of teams that propose short contracting projects to candidates to try them out, because I would never agree to interview on those terms.
Amen. "Contractor for two weeks" and "full-time employee for the first two weeks of a years-long job" are fundamentally different positions. They require different sorts of deliverable [1], different contracts, different social positioning relative to the team and the company [2], and (of course) different pay rates. To pretend that one of these positions is a special case of the other invites trouble.
I'd say more but your earlier post said it better: https://news.ycombinator.com/item?id=4559389
---
[1] My rule of thumb is not to accept a "contracting" gig, even a "temporary" one, that doesn't have more specific deliverables than "show up and work X hours per week while looking competent". Work that is measured only in hours is not contracting: It's employment with additional risks and no benefits.
[2] The team knows what a contractor is, and they know what a full-fledged fellow employee is, but the arrival of someone who is neither fish nor fowl tends to sow hesitancy and suspicion. I've seen what happens with "probationary" employees: The label never quite fades away. I remember one fellow who turned out to be a real star: At company parties people would say things like "hey, I remember when we hired X even though we were worried about his experience level, but he turned out to be really great!" This always struck me as a painfully barbed form of praise.
Hire like you mean it. When you add someone to the team, add them to the team.
Also, I've been interviewing. The experience bloody sucks. For example, and this is my hobbyhorse, graphs/trees are stupid. I've written a graph algo more advanced then binary search exactly once in a 15 year career. I got in an argument in the last interview because I knew cache clock costs for a Nehalem (if you miss and go to main ram, 198 cycles) and argued this meant you should always, if possible, use a hashmap. Anyway, I still get judged on my ability to write graph algos on a whiteboard. Sigh. So I have to study this nonsense for two weeks before every interview then proceed to entirely forget it again.
There are also enough similarities that I'm surprised that you're so opposed, especially since you kind of dangle an interview offer at the end of solving the whole crypto challenge. From your side of the table, what are you looking at from people who finish the crypto challenge?
But.
Developers that do interviews seem to want a step by step algorithm for doing an interview... Or worse yet just want to throw them a topcoder style algorithm question so they can "spend" an hour on the interview and come up with an easy yes/no answer.
I've been conducting interviews for about 5 years and started out doing the same. I came to realize however, that we were rejecting perfectly fine candidates.
Several devs would either knock our take home puzzle out of the park or show us a great portfolio, but then would flub or stumble when the (mostly JR) devs would ask them to implement something akin to a topcoder challenge.
Now, much like we expect devs to tailor their resumes to us, I tailor my interview to the person being interviewed.
If you're relatively new, you might get asked some data structures / algorithm questions.. If you're senior expect to go into great detail about your past projects and design decisions.
There is no process or formula. Interviewing is an art.
I think any coder will be more comfortable doing real job instead of studying college stuff that you haven't done in years.
(1) Different hiring practices and their long term success rates
(2) Different procedures (agile, scrum, cubes or no cubes) and whether they impact working efficiency
I just feel like everyone and their mom has these grand ideas about how they can do things differently than everyone else but I never get to find out objectively if they work.
I doubt you will get great developers that way.
> I’ve filtered through resumes and listened to coworkers...
and then...
> ... which is why we don’t ask for resumes.
I am confused.
I can tell after talking with someone for 5 minutes whether have the right stuff or not. You can tell by how they choose their words and express their opinions.
The best interview process:
1. Candidate sends company a resume.
2. Company reviews resume and likes what they see
3. Company calls candidate for a quick 5-10 minute call.
4. Questions asked are "What are you currently working on/thinking of starting" (referring to any kind of extra-work projects the candidate might have), "what is your favorite language and why", "what is your favorite library/framework and why", etc. Like I said before, its usually pretty obvious who comes from experience and who is novice. For instance if someone said they prefer PHP over C# because "PHP is faster", you can assume that person isn't the most experienced... The other day I sat in an interview where the candidate was an older guy (grey haired who graduated in the early 90s). Not even 25 seconds into the interview and I knew immediately that he was the read deal. It was something about how his stories seemed to have the right details. The problem is that it takes experience to judge experience. When I was 22 years old (which is the average age of most startup founders these days it seems), I would have been more likely to dismiss him as an old grey haired coot whose skills are outdated.
5. Company decided candidate is experienced and invites the candidate to a face to face. This may include travel expenses if the candidate is not local. At this point the company pretty much makes the decision to hire the person.
6. During the face to face, the objective is to give the candidate a clear description of what the company expects in terms of work hours, salary and benefits, etc. Also, what kind of problems they are working on, what kind of technology they use, company culture, etc.
7. Candidate and company negotiate, and then eventually agree on terms of employment. Candidate starts whenever possible.
The problem is that it takes a good developer to know another good developer. If you are a non-technical person, or a low-experience developer, you don't have much intuition to go off of. Instead you have to resort to making candidates jump through hoops by making them do things like FizzBuzz.
Another thing I should say. Interviewers should focus less on judging the candidate based on personality. The way I see it, as long as you're passionate about the right things, I don't mind if the person is prone to jerk like behavior every now and again. All brilliant people in the history of mankind have been described as "jerk" at one point or another. In my experience, the truly talented will always set aside their sometimes giant egos in the end for the benefit of the project. Its kind of like Justin Bieber. Despite him getting into controversy all the time, he still manages to get out there every night to put on a show that makes the fans keep coming back for more. The day his promoters stop putting up with his crap, is the day he fails to draw ticket sales.
In other words, more companies need to stop basing their hiring practices on repelling people prone to "Beiber behavior", and need to start basing it on bringing in people capable to drawing "Bieber crowds".
I think the reason this process appears to work is that all the other conventional hiring processes are equally bad; if you're shooting for "reject obvious crazies and incompetents and then accept statistically insignificant deviation from random chance", sure, choose the technique with the least overhead no matter what.
Is there a better bar to set that's actually achievable? I think maybe our expectations are set too high here.
In online marketing, the objective is greater sales. That's a pretty legible goal, captured neatly into the idea of a funnel.
With hiring, it's not so clear cut. What metrics do you set? What are the metrics supposed to get you? There are so many variables that it seems like any system you use will be garbage in-garbage out.
Which metrics? Depends on the role. If you can't tell me what makes a good developer for your team, how can you tell me you're managing that team? The answer is: of course you could tell me what a good developer is, push coming to shove; it's just an annoying and complicated question to answer, so we all tend to substitute an abrasive and counterproductive interview process for that answer.
Uhm, I graduated in 91 and am 42. If that's considered an older guy, then I can see why there are so many hiring problems.
Nothing says how old the guy was when he graduated, or how far he went in academia (i.e. Masters? PhD?). It's also possible for people to have greying hair in their 40s, which may lend to someone looking older than they are.
All the research done on interviewing tells me that you're wrong. But you're unaware that you're wrong.