Show HN: Takehome.io – Time-limited coding challenges for interviews via Git
takehome.io
takehome.io
For code exercises I set when recruiting I send a pdf out and then expect a zip file or link to GitHub or bitbucket with the solution. I might give an advisory along the lines of this shouldn’t take more than half a day. But if the candidate screws up and starts again that’s ok. Mistakes and wrong turns are taken all the time in the real world. It works very well. Simple. No complications. And free.
The idea is that the candidate knows they can do the best they can within the time limit, and then it's done. No agonizing over whether it's good enough or they should spend another few hours. No imagining how many hours and days other candidates for the same role might be putting in, wondering if they're going to stack up.
I've used this approach with around 100 candidates hiring for various roles in my own business and for clients. The response I've gotten from candidates has been overwhelmingly positive.
The problem I have with the traditional "don't spend more than about X time" approach is that it's not a level playing field. The responses to a challenge exist in relative terms, not absolute. It doesn't matter whether your response was great or not. It matters whether your response was in the top 5 of all the other candidates responding. It follows that you probably need to spend at least as much time, if not more, than all the other applicants for the role. How much time the other candidates _might_ be spending is basically unbounded, leading candidates to spend vastly too much time working on a challenge and then lying about it.
If the candidate screws up a timed challenge that's also totally okay. It'll be a great conversation to have in the follow-up interview. "I started down this road, but if I was doing it again I'd do X, Y and Z because of A, B and C."
Incidentally, I'd suggest that half a day is way too long. Especially if that half a day isn't actually enforced. When I'm hiring I usually set challenges of about 40 minutes, an hour and a half max. I'm not looking for a product that's ready to ship to production. I'm after a sense of whether the candidate can actually interpret a problem then string some code together, and something meaningful we can discuss in the next interview.
While I agree with what you said about leveling the playing field, I find 40 minutes sound awfully short. In university I noticed I was doing much better on 3h exams than on 1.5 hour exams, often only taking 1.5h for the 3h exam, even though it had more material. I couldn't relax enough in the short exam to do my best, even though the time was long enough for a twice as long exam.
Same here. I was the person who would do poorly on the 15 min physics quizzes we had in college just to get the highest grade in the class on the final, only because I had more time (half the answers on my quizzes would be blank just because I didn’t get to them). In high school, I would barely squeak by on the multiple choice AMC to then do fantastic on the 3 hour AIME.
But I clearly knew the material, and I knew it very well, so are you going to penalize me for being slightly slower on a test? As an employee, I complete my real world tasks just as fast as everyone else, so why should a time constraint on a high pressure exam relate to the real world productivity and profitability of an employee? I would argue that it doesn’t—at all.
I suppose it depends on the employer, does he prefer candidates that put in more effort but might be less talented, or very talented but might not have much resolve (unproven at least).
If the employer is being honest, then we're all good. But if they're going to go with the candidate that puts in 5x the time allotted, then I clearly don't want to work for them either.
It's an obvious red flag IMO. If they ask for n hours in the interview but expect more than that, then they're likely to do the same thing when you're on the job. Don't be surprised if a few months down the line you're working 12 hour days 6 days a week or you're an underperformer and the first to get cut.
You're actively removing the possibility of finding out who would think "meh thats good enough I guess" is the effort needed for production code and people who would go the extra step of spending time analysing and fully solving the problem.
2 people with equally average solutions...would one have done more given the chance? Both? Neither? How can we find out? You've added uncertainty to the process rather than solved any problem.
Few people perform well under pressure. Do you want to test what is their mediocre performance under time limits like on a school test or what they are really capable of?
With this approach you will hire people able to churn out crappy but "good enough" code in the limited time. Not people who actually can do quality work.
> Few people perform well under pressure.
Yes, but majority of jobs have some level of time pressure. That being said, there is such a thing as reasonable deadline.
> With this approach you will hire people able to churn out crappy but "good enough" code in the limited time. Not people who actually can do quality work.
Inability to perform under pressure does not imply high skill without pressure.
Ability to churn out crap in a time limit doesn't imply competence neither.
I do think an interviewer should be completely up-front about what they want out of a candidate if they give a coding challenge that's too ambitious to implement perfectly within the time limit.
So, I guess that companies which administer such time based tests foresee at least one such situation for that role. I'm just speculating.
There are quite a lot of software jobs like this. Maybe the time pressure is more 2 weeks instead of 4 hours, but deadlines are a real thing.
In my job, pretty much every week. Many of my tasks are "implement feature X to a passable amount in 2 days", and then it goes to someone to use and get feedback on immediately. My team are mostly the same.
No. Forty minutes is far too little time. Many people take 10-15 minutes to even get settled into coding. I can't imagine a meaningful, applied, realistic exercise that would take 40 minutes, unless you're wanting basic data structures like a linked list, or a basic sorting algorithm, or something.
You're missing a lot of good people that way, but I guess that's just collateral damage in the quest for the perfect hire, right?
Both methods are ways to tilt the control and convenience back in the favour of the company - defeating the whole purpose of doing the work at home.
With all due respect, but attempts to optimize the interview process invariably end up doing this, due to the priorities of the paying parties (ie - the companies) - and always at the cost of the interviewers. No thanks.
But lots of people (myself included) get debilitatingly stressed out when an arbitrary clock is put in front of your face. Honestly, my recommendation to any company that gives timed challenges is to... not.
What if you gave the option to not time challenges? And maybe didn’t make it seem like having untimed challenges are inherently more stressful all the time.
I like the idea a lot, but I think it is up to the team and hiring manager and not the tool the decide whether to use a time box or not. I haven't tried the tool yet so I'm only speculating here, but as a potential user from the team / hiring manager side I would expect I could disable the time boxing completely if I wanted.
Cheers!
>The problem I have with the traditional "don't spend more than about X time" approach is that it's not a level playing field.
No, it's not. And that's actually fine. I don't care if a candidate took five minutes, or five hours, or five days! In fact, the response time can tell you something about a candidate. Don't take that away from me as a hiring manager. I've actually never asked or been asked how much time I spent on a challenge. It's not an issue to solve!
>If the candidate screws up a timed challenge that's also totally okay.
What follow up interview? They fu^ked up the challenge. You really think a hiring drone is going to proceed that candidate when they have 10 others who did ok? That goes completely against what this sort of canned test is all about; getting past a hurdle. If hiring managers were going to interview all candidates who took the test and have a nice chat about their code, the test is completely redundant. They'll just ask for a code sample to talk about. Saves everyone the time and the money!
All in all, 40 minutes is way too short and artificial cut offs are a nasty way to try and "level the playing field". In fact, you're not leveling it at all! Just making sure candidates who may be taking a bit longer to get into the swing of things that day are discounted. Heck, if the candidate has simple network issues they're screwed!
which is unlikely to be available if they aren't the owners of the IP.
So either you have to guess real lucky, or have a probation period of a week or two with any new hire. Wouldn't you say that's worse than a short take-home project (for the employee)?
Of course, unless it was already on Github, etc.
[while I'm in the "there's no secrets in software" camp, there are still policies and laws to follow]
You've possibly eliminated a lot of adults who are 10+ years out of college and have family commitments.
You've possibly eliminated people with significant side-interests outside the realm of software. Writers, artists, musicians, marathon runners, activists, etc.
Just because someone doesn't have side projects doesn't mean that they don't have ambition, enthusiasm for learning, or other ways to show you they're perfectly capable.
I think the "more than just a 9-5" idea is a toxic pattern which has been championed by the privileged. In short, by wanting something like this, you risk selecting for a fairly homogenous set of people.
I think you can write a good coding challenge which doesn't look like all the others (the socialite example in Takehome.io is particularly good because it scales with skill level – it's not a pass/fail, but a spectrum from just making it work, to showing you have thought about and dealt with an array of odd things that might crop up).
The biggest reason for this attitude stems from the immense amount of people who are getting into CS just for a paycheck. Sure, we're only working to get paid, but in my experience, these people have zero motivation outside of it. Endless hordes of bootcamp type people, who see six figure salaries, and can spew out all of the stuff they've memorized in a book. Without actually thinking about why or when. I'm not looking for "rockstars", "ninjas", "gurus" or whatever other nonsense words phrases use, but rather people who are genuinely interested in their field.
For example, writing up a silly application to make a daily task easier. One candidate had their own tooling for time tracking. It wasn't much to look at but it was there and working for them. It wasn't on github, well commented or entirely complete. It was however a neat tool and well enough written. The candidate didn't really think about it as a project, just a little tool they wrote.
To take it all the way, I think the follow-up discussion should be done strictly in text, using some combination of IM and a web-based code review tool. So the interviewer would call out parts of the PR and comment on them through the code review tool, and the two could have more free-flowing discussion over IM. Again, this would be just like the way it's actually done on the job, at least in my dream all-remote environment.
As for fairness, I'd argue that timing it is the only way to ensure it. This way you know that the playing field is level between you and the other candidates applying for the role.
To fun, I'd say that the least fun thing I can think of in this context is spending days wondering whether I had "done enough yet" in order to make my submission stand out ahead of the other applicants. Much more fun to know that as soon as that clock ticks over, what's done is done and I can get on with my life for better or worse.
Also, all of the existing sites I've seen have one or more of the following problems:
* Make you use their editor * Make you install their software on your computer * Make you use some specific language * Make you use their set of challenges
Takehome uses nothing but git. Whatever you put in the repository and how you get it there is up to you.
That holds true both for the candidate and the challenger. Takehome encourages challengers to create their own task matching the kind of work/language/framework/etc that's relevant to their team.
In my experience on both sides of the desk, a more holistic approach yields better results in the long run. A resume scan, then a review of presented code, then a phone screen to determine communications ability and check some boxes, then a short series of in-person conversations with the team to determine "can we work together?"
And for so little done by the service, I find the cost to be completely perverse.
If you're issuing more than 5 challenges per role you fill something somewhere has probably gone awry. $100 per role hired is no more than a rounding error.
Also, the timer starts from the first `git clone` rather than the first push.
Also doing interviews before technical challenge is a sure suicide for companies. In my experience to minimise the waste of time you send out quick and easy technical challenge (no more than 30 minutes!) and you filter out 90% of applicants. With remaining 10% you have a phone call and if you have more than X (magic number of 3 in my experience) of qualified applicants you send out more thorough tech challenge (no more than 2h long!).
Rinse and repeat until you get to X applicants which you will invite for face to face interview.
Truth is that almost no-one in tech is qualified to do face-to-face interviews. Either they are not on the level technically or dont have the communication and HR skills. Interview in person is mostly to see if you are not dealing with a weirdo and if that person will be a good fit for the team.
A side note, I tend to issue valuable response to every candidate. If you do that, they are very likely to re-apply, step up when someone drops out or recommend to friends. People tend to do/say the same dumm stuff so you can prepare those replies as well.
Are sample questions provided? Offer good quality questions and a monthly rate for a certain number of candidates. In some organisations it's easier to approve a monthly expense than a per use expense.
You should probably check the user can git push before starting the timer.
In most of the Europe take home exams are common during the interview process, but your online test should be 1h (at most 2h). If you make it longer, very few ppl will invest that much time in competitive job market such as software engineering.
Take home tests are way harder in the hottest market such as San Francisco Bay Area. A lot of candidates will simply drop out of your pipeline if you send them. IMO there are good for fresh grads, junior candidates or candidates that would you otherwise reject in resume screening. There are hidden gems, often in underrepresented minority that simple don't have anything impressive on their resume yet. Being able to identify them can be your competitive advantage.
The tests are tidy and the in-browser IDE is not bad.
With coding tests, we isolate a smaller problem and send that as a fizzbuzz screener. This scales well as solutions are checked automatically.
When I was recently job hunting, I had a bad experience with a company that wanted me to take a timed coding challenge where I would clone a repo and then submit a pull request with my solution. What started the timer then was them opening the repo to me, not me accessing it. This required us to agree on a time for me to take the challenge. Long story short, the recruiter proved incapable of opening the repo at the times we agreed on.
Avoiding that need for coordination and chance for the recruiter to mess things up is a nice benefit of your service.
When I'm choosing a time I'm actually less worried about the complexity of the exercise than how much of a candidate's time it's reasonable to ask for. If the challenge is too complex to be done in 40 minutes or so, I make it less complex.
I also put language in the intro text like 'If you don't end up with a complete solution in that time, that's fine, just submit it anyway. We don't expect this to be enough time to write "The Most Perfect Thing".'
If you're curious, sign up and take a look at the "Socialite" sample challenge.
Also, it's basically just 2 forms and some scripts.
I would choose a large time window- 4-8 hours or so. That way it ensures there is plenty of time to complete the challenge but also that it gets done in one day, like a normal workday.
Yeah, if they've used git before. Otherwise they'll need at least a solid 30 minutes to figure out the most basic git usage pattern.
Do you really care about what source control system a person has experience in? Why? Any competent developer can pick it up quickly enough.
Do I need to use another ATS ? then my team will just give up on this. If you have a limited ATS in there (with stuff like emailing from the dashboard, reading replies from the dashboard), then this is killer
Honest question: why? Can't your card / bank / PayPal do the conversion for you? (I say this as a Canadian who often pays USD with my Canadian card / bank account).
We hire a lot straight out of college, which I think is probably where this type of test is more common.
Personally this is how most online services should be, pay per use.