How to politely decline a take-home test task
workplace.stackexchange.com
workplace.stackexchange.com
> Currently, I'm about 2 and a half hours in, trying to get my virtual environment to run the baseline code they provided.
Reading between the lines (and from the comments/follow ups) I think the take-home challenge might be working as designed: Filtering out candidates who are not yet experienced enough to handle the type of work the company does.
The author’s desire to substitute some of their own, different, code samples as evidence of qualification is likely missing the point: Quickly getting the code to run in the provided environment was part of the challenge.
We had a similar problem at a past company where we cast too broad of a net for candidates: When we did no-prep trials of the take-home challenge among team members who hadn’t seen it yet, completion times were all under one hour. When we gave it to loosely-screened candidates, we got several complaints from people who said they were on hour 8 and still couldn’t make a lot of progress.
Tightening up our pre-screening mostly fixed that problem. It’s not good to waste candidate’s time if they’re not going to be qualified.
Every single take home test I have ever taken. It took multiple days to finish. And even after finishing it, you still don't get the job.
In one place I not only finished the test. Their onsite interview was made up of four more rounds. The first two were about implementing two features on the spot where two of their engineers watched as I wrote the code. And then in the third round they review your code. All onsite. I not only finished the original assignment, added the two features requested and substantiated every single query on code they had in the code review round.
Yet in the last round, they rejected me because I didn't read their employee's favorite paper. And therefore must be a bad programmer. And this is the one of the most famous company in Bangalore which routinely brags about hiring programmers only on the basis of take home tests.
The whole take home test thing is a giant fraud. Don't fall for it. They are not serious about hiring you. Serious people know whom to hire after an hour or two of discussions.
Also contrary to whatever people think about the archetype inefficient programmer who fails at fizzbuzz, most people can actually code. This whole take home test thing is a marketing campaign and generally a way for in house programmers to make themselves feel superior by routinely asking candidates to finish projects in impossible timelines and yet reject them like they are gods.
I mean, not in this case -- in this case they got 3 features implemented for them for free. That's why they did it.
I wouldn't take that at face value. It probably isn't the real reason. People often hide the real reason for rejecting someone and will use some lame excuse instead.
Either way it shows poor judgement on the part of the company.
I've also designed at home tests. I tried to keep them under a half-day and invited people to send me something from a portfolio of past work instead if it corresponded. Work samples are good info. Work samples that are time efficient are even better.
Still, I've also had the really negative experience where I was sent tests that seemed poorly conceived or overdemanding and had no idea whether my work was evaluated or if so how. This is how I tend to do things now:
* If I haven't had a serious interview/screen from someone beyond a recruiter I'll do short well-defined tests (< 2 hrs, or about the amount of time one might expect for an early interview). If I'm asked for something longer, I'll ask if we can wait on that until I've spoken with the hiring manager or a prospective teammate and we both feel like things are plausibly promising.
* I ask if anyone internal has completed solutions and what their estimated time spent was. Also how many external applicants have have completed it and what the range of time invested by candidates who "passed."
* I ask if they have an evaluation rubric and if they're willing to share it with me (along with completed solutions) when I'm done, and what they hope applicants would do if at any point they're stuck.
If the answers to these questions seem well-considered and promising (along with other signals about the job), I'll invest in the test. If they seem poorly-considered or worse evasive, I'll think twice or politely pass. Good answers don't guarantee a positive outcome, but they make it more likely it'll be at least productive as an exercise.
Only if you mean it in "everyone can type 200 wpm" kind of sense. If most candidates can code in your panels that most likely means recruiters/hm preemptively rejected every candidate that doesn't have years of experience at faang or whatever local equivalent is ;) (source: did 300+ coding interviews)
What's strange here is that you're coming this conclusion without having actually ... seen a description of the take-home task itself.
By all appearances, you are basically assuming the company was right (and knew what it was doing in designing this "test") from the get-go.
My own observation indicates the opposite: not always, but far too frequently these companies either just don't describe the task properly; haven't checked if the components / targets actually still work with each other; or just plainly overestimate the time it takes to do the steps the first time, out of hubris (the way clients, managers, and developers tend to wildly underestimate the time it takes to do just about anything).
Which allows them to chortle to themselves afterward: "See, all these candidates failed or ghosted us after agreeing to take the test. That must mean it's a good test. And also that we must be awesome, for having such a hard time finding candidates that meet our exacting standards. But with sea of bozos and wannabes out there, what are you gonna do?"
If you don't want to do the take home test, tell the company just that. You aren't going to hurt anyone's feelings. You don't have to be overly polite about it or think of excuses. No one there is going to care.
They will just move on and hire someone else. You can move on and apply somewhere else. That's all.
(and evidenced by "I will also state that they can check out some of the previous code I have written on my GitHub." where the candidate still wants the company to keep considering them after declining the test.)
How do you know that's the goal of the test?
Ask yourself how the company founders and initial employees got their jobs. Almost certainly not by going through their ex post facto, arbitrarily selected in-house hiring process. Was that unethical?
IMO there's actually nothing wrong with hiring the first qualified applicant who shows up, and then calling off the search. Is there an ethical obligation to hire "the best possible candidate", and why? See https://en.wikipedia.org/wiki/Satisficing
Is it unethical to recruit an outside person you already know is good, and not offer an open position for everyone? It it unethical to go with an internal candidate and not offer an open position for everyone?
Companies tend to waste way too much time, effort, and money on hiring. It's premature optimization, the root of all evil.
Uniformity is not the same as ethics. Otherwise, we'd be ethically obligated to eliminate special parking stalls for the disabled. Moreover, there's nothing inherently ethical about any given company's uniform hiring criteria. Any given set of criteria may "discriminate" in favor of certain traits and against other traits. How do you judge the ethics of the criteria themselves, in comparison to other possible uniform criteria?
For founders this is a perk of being a founder. You are welcome to found your own company if you wish.
> Is it unethical to recruit an outside person you already know is good, and not offer an open position for everyone?
Yes, it can be. Especially in larger organizations that have policies around diversity and equal opportunity, it is in fact a requirement to have a large enough and representative pool of candidates.
Yes, but that didn't answer my question about ethics.
> You are welcome to found your own company if you wish.
I did!
> Especially in larger organizations that have policies around diversity and equal opportunity, it is in fact a requirement to have a large enough and representative pool of candidates.
How often do tech companies get a "representative" pool of candidates?
But this actually goes to the other issue I raised:
> there's nothing inherently ethical about any given company's uniform hiring criteria. Any given set of criteria may "discriminate" in favor of certain traits and against other traits.
In particular, take-home exams that take many hours to complete tend to discriminate against certain kinds of candidates, those who don't have a lot of free time outside of work, family, etc.
It's important to note that these tests are primarily used by tech companies as a screening device, not as the method for making the final hiring decision, which tends to be more subjective. They don't simply add up test scores and choose the highest scorer.
Moreover, the tech industry seems to be obsessed with tests, but they're an outlier. Tons of other industries use more informal methods of hiring and don't give tests at all, much less take-home tests. Do you think they're all unethical?
On the topic of hiring, a "Job Simulation" is one of the most objective measures, which is pretty close to what a practical test is. Questions about things candidates actually did is another important technique (e.g. tell me about a time you ...), and the third category of techniques that work is giving hypothetical situations that they need to explain how they would respond to.
Although informal conversations are common in industry, they are some of the worst techniques. For example, if you start with a "coffee chat" research has shown that most people more or less make up their mind in that first meeting then bias the rest of the interview results to unknowingly justify their decision.
> If you care at all about bias and diversity, you may want to learn more about how to construct interviews that reduce both.
Interviews, whether formal or informal, are a very bad way to evaluate the skills of job candidates. The best predictor of future performance is past performance; research the candidates as much as possible. But in any case, hiring is a crapshoot.
Interviews are useful for getting to know each other, seeing if you want to work with each other, and asking clarificatory questions.
> On the topic of hiring, a "Job Simulation" is one of the most objective measures, which is pretty close to what a practical test is.
Tech sector job interviews assess anxiety, not software skills. https://news.ncsu.edu/2020/07/tech-job-interviews-anxiety/
https://www.eeoc.gov/laws/guidance/employment-tests-and-sele...
"I know that every other candidate for the job successfully did this exercise but I found it too difficult to even set up. Can you hire me over all of them?"
What do you think the company's response will be?
"Every other candidate was desperate and rejected a thousand times so they're happy to waste their time on your pointless exercise that you won't even look at."
The good candidates are busy getting hired by companies that understand how to evaluate their work based on interviews, references, and several other things that don't waste their time.
But there were also reports on HN that before the layoff wave, they'd get 2-3 applications per year for a senior position. It is of course any employers right to continue applying the same hiring standards as eg. FAANG/Big Tech would. But if I were the employer, a better strategy in such a situation would be to just swallow that I can't pick and choose if I have nothing to offer for leverage :)
I'm sure you're smart enough to realize that what I'm hinting at is that the job market is a _market_ and that employees don't have to put up with every bullshit depending on the market conditions.
If you accepted the take-home, and failed to get started on it, and then asked for something else, I wouldn't. Because I have evaluated other candidates and they were able to do it.
I don't do these sorts of tests, personally (I even had one that they expected would take a full 40 hour workweek to complete!). If I'm presented with one, I know that in declining to do it, I am disqualifying myself for the position.
It's just life. We all decide what our limits are, and in deciding those limits, we exclude some possibilities.
They spent longer time than preferred.
And yes, they're looking for a shortcut.
There is no way around fulfilling the task.
Not revealing the time spent poses risk of impostor syndrome, risk of actually landing a very stressful job.
Refusing to solve the task indicates that they would rather post-pone finding out, and rather risk rejection if it means not being challenged on their solving speed disqualifying them, because they think that it too likely would disqualify them.
Personally I prefer a take home over a full day technical whitebording on-site which is my idea of hell. But unfortunately many companies do both in the same loop instead of one or the other
About a month ago I had this happen:
-Explained full interview process during the phase 1 call (15 min call, round table, and lastly a take-home code challenge)
-they were cool with all this
-they get to the end of phase II, after an hour long call with a bunch of people, and passed it/went great
-they then surprised us by refusing phase III/the take home test
As an employer I try to set expectations early, they acknowledged it and then proceeded to get upset when we did what we said we were going to do.
Also, why would you do a round table before a take home? Do you not value the time of your people?
I mean they made it through Phase 2 and sorta 2.5 where we confirm salary and they accepted. Everything seemed on track until the test was mentioned.
If they ghosted us after the round table interview then sure, but why come back and accept a salary just to bail at the last step?
If it were my company, I'd be taking a cold, hard look at that process to be sure that it's OK. Perhaps it's not really going as well as it seems.
I suspect that your messaging during phase 2.5 is bad and you're misleading candidates into thinking that you've decided to hire them, and then when you follow that up with a take home they conclude that you're fucking with them or a disorganized mess.
By doing it at the end, you are just telling the candidate "Oh, you are good. You passed phase I and phase II... but that's not enough you poor candidate!".
Every time I've been presented with a take-home test, it was done this way. I'm genuinely surprised to hear that some companies do this at the end.
Basically, assume that candidates can drop at any time.
My advice:
Your first step as a candidate when given an assignment (with specific time constraints that you feel are mis-estimated), is to guestimate what you think it will take. Remember, this is exactly what happens at work with task estimation. Are you going to 'politely quit' if your manager gives you a task that can't be done in a certain time window?
"I reviewed the task and my estimate is that this will take a few more hours than specified. I would like clarification if the time limit is an informative hard limit."
One benefit of asking for estimate confirmation (specially for people like me who tend to go for more general solutions) is that you may actually get useful feedback. If you get a firm 'yes' from a competent team then it means they are looking for absolute simplicity in solution, and your weekend project of pimping your take home task is not only a wasted effort, it may actually take you off the list.
Whatever response you get back it will be informative, sometimes even shedding light on what sort of organization you are applying to join.
What? I like sokoloff's response[1]. It takes 3 seconds.
Just a guess, but perhaps they're trying to provide some constructive feedback to the company to let them know that the time estimate they're giving to their candidates is potentially way off. (That said, maybe it's not way off for people with more experience, but the candidate doesn't get that.) It's probably futile, but I can understand the urge. If they company doesn't know a problem exists, they can't fix it (if it is an actual problem).
I worked for a company that administered a simple half-hour take home coding challenge before any interviews. The problem was simple but not trivial — print Fibonacci numbers, take a string of brackets and print if they were properly nested (e.g. '()(())' is OK, '(()(' is not).
We had essentially zero attrition of candidates with good CVs who refused to take the test. On the other hand, we disqualified >80% of applicants from the test alone, saving an enormous amount of interviewer time. It would have sucked to have phone screened all those candidates and spent half an hour of company time on each one to discover they can't code at all.
I'd be even happier if it replaced a whiteboard FizzBuzz, and would raise good flags if it was a mathy problem like the ones the OP talked about, instead of unmotivated terminal or memory juggling.
I'm nowhere near elite...but hopefully none of your candidates need anywhere near 1/2 hour to code stuff that simple.
Usually it's a "Please write a program to solve this problem with the following rules" or "Please write a service that services x,y,x"
And so on...
IDK, is it that simple for everyone in tech to finish it in way less than 30 min? I think I would definitely need more than 30 min for the nested brackets validation problem and I've been in SW dev for 10 years and I'm about average in my area compared to everyone I know (non-faang/non-big-tech).
The thing is, I haven't touched algorithmic riddles like that since I graduated university over 10 years ago, and none of my daily assignments at my coding jobs since, were ever anything close to that, and nor do interviews in my area (Europe) ever require those kinds of problems to be solved, so I would definitely need some time to warm up, grab a piece of paper, and think of a solution for that one.
New-grad version of me who did a lot of algorithmics assignments, lab work and interview prep back in the days could definitely be faster, but this version of me who's head is filled with solving the issues assigned at work, not so much, and would need longer.
When I apply to jobs in my area, most companies here just check if you have domain knowledge related to the problems you'd have to solve if you got the job. Coding riddles like that seems more of a US, big-tech, VC-unicorn kind of thing that's missing in the companies I applied at. I wouldn't mind such riddles as an interview, but I definitely would fail them if they were strictly timeboxed.
I guess I'm not employee material for his company.
Increment the count when you see an open bracket. Decrement the count when you see a close bracket. If the count every goes negative there was a mismatch. If the count does not end up at 0 when you've processed all the brackets there was a mismatch.
It can also be treated as a text transformation problem. Every non-empty string of matched brackets must contain at least one '()', and deleting a '()' from a string does not change whether or not the string is balanced.
That gives an approach of repeatedly finding and deleting '()' from the string until there are no more '()'s to delete. If that gives you an empty string the original string was all matched. If the final string is not empty then the original string had a mismatch. Here's that as a shell script.
#!/bin/sh
S=`echo $1 | tr -c -d '()'`
P=x
while [ "$S" ]; do
if [ "$S" = "$P" ]; then
echo "Mismatch"
exit 1
fi
P=$S
S=`echo $S | sed -e 's/()//g'`
done
echo "All match"
exit 0IMO:
({[]}) // has balanced brackets
({[}]) // has imbalanced brackets OpenCount = 0
until( Exhausted( input ) ) {
if( NextChar( input ) == '(' )
OpenCount++
if( NextChar( input ) == ')' )
OpenCount--
if( OpenCount < 0 )
ERROR( "Negative Nesting Level in Input" )
IncrementPointer( input )
}
if( OpenCount != 0 )
ERROR( OpenCount." Unclosed Nesting Level(s) in Input" )
Print( "Nesting okay in Input" )
Could be improved as a collect.. but that was a first pass. [Did take some go overs to fix a few typos]
I have no idea how to calculate Fibonacci numbers.
I'm a decent coder, have a bachelor of Software Engineering but never "loved" math.
I'm sure I could learn in a three minute Google.. but I certainly couldn't do it on a whiteboard with no Google access.
1 1 2 3 5 8 13 ...
That's it. Now you know.
It also generalizes, apparently "Fibonacci sequences" are defined by the initial 2 values, with the rule mentioned above. I thought this was weird once but upon checking it it is true that, for example, 4 5 9 14 23 37 would also be Fib{4,5} or something. Idk it's been a long time.
If they look for full stack dev and then grill on some obscure details of one tech stack piece it is also their problem.
Is company looking for experience like "someone who could connect to k8s and debug application"? Is company looking for "someone who can setup k8s and have networking up and running"?
How do deal with people claiming "experience with" where someone set something up for them and they were using stuff but never had to dig into details.
In the end it is also like driving - while driving is driving then "F1 driving" is so much more and while F1 driver is not an engineer you still expect him to understand how ICE engine works and bunch of other technical details. For everyday driver you expect no more than "put in fuel, turn the key, throttle and you drive, break you stop".
Staff: Create and deploy the app with a CI/CD+deployment platform. (K8s)
We only do it for certain demographics (i.e. interns and fresh grads) while experienced candidates skip it, and it certainly saves a TON of interviewer time.
Even for the internship alone we get tens of thousands of applications and it's pretty intractable to review all of their resumes manually in a relatively small window of time, so there needs to be some type of automated system to discard a majority of candidates.
* code review: Screen share with the candidate a small code snippet in a language/framework that they are comfortable with that needs improvement. Have candidate talk through what feedback they would provide in the context of a code review to improve the code.
* code pairing: Pair candidate with an engineer on the team to improve some existing code (add a simple feature).
Both of these approaches have been very helpful at understanding how skilled a candidate is, how familiar they are with a language or framework, how they provide constructive feedback, etc.
Code snippets can be designed to encourage discussion of security practices (SQL injection, for example), capacity planning (5 transactions/day, /hour, /sec?), algorithmic complexity, etc.
2.5hr total, then moved on to next in-person round. Their in person interview questions are all based on either my code or their server side code for this test project. It was great because I already knew about it from the take home.
They explicitly asked for a simple, working cli program in the language they use, in a reasonable time box. They said if you have time, consider test-ability, consider performance (the service streamed numbers so you needed to not store them in-memory), and other things a “real” program would need to consider. They gave themselves amble signal for a simple project, and set the time box low enough that most anyone can find that time.
I will absolutely be using a similar process for hiring at my current company.
const test = toTest => { try { new Function(`return [ ${toTest.replaceAll('(', '[').replaceAll(')', '],')} ]`)(); return true } catch (e) { return false; } }
I would agree that companies that give "2-3 hour" take home problems are a red flag for me — red as in, I won't do it anymore. That situation becomes an arms race quickly: if I can solve it in 3 hours, why not spend another 3 hours doing a second, more elegant solution that makes me look even smarter? I'm sure that's what the other candidates I'm competing with are doing. And if they're spending 6 hours, I should spend 8 hours at least, to get an edge over them. But they know I'm spending 8 hours, so they're probably spending 10...
I wouldn't say simply having at 2-3 hour take home by itself is red flag. But if the take home is a pile of hot garbage (poorly or inadequate documentation on how to setup a local env for example) then that is red flag.
My first hire (that wasn't someone I knew/previously worked with) I did not require that they complete a take home test. That hire turned out to be a complete disaster. I have since started requiring take home tests, and I will never again hire anyone that I don't know without a take home test. After requiring take home tests the candidates that have been hired have been much better.
It doesn't have to be a docker run.
>I can think of several well documented setups that can take upward of 2 hours
If ALL you have is "well documented" setups that take 2 hours, then that company sucks.
Look you don't have to have docker, but if you have a setup that is merely a Confluence page, or a pdf file, and don't have ANY shell scripts, or any way whatsoever to bootstrap setting up your environment locally, then that is a massive red flag.
"It works on my machine!"
Granted, sometimes fluency in the XYZ of their stack matters - but usually it just doesn't, and grilling people for this fluency shows a lack of sense for the broader picture, in my view.
https://versastudio.com/blog/if-carpenters-were-hired-like-p...
1. It is near trivial for an experienced programmer in that language to do the task: You don't belong applying for this job. Example: Python. If you can't setup a python virtual env in about 30s... don't apply for a python job.
2. It is near impossible without their IDE settings. Java projects can be like this, at times. Though usually not to the tune of 3 hours, but I could see it getting there with a complex J2EE + database setup.
Know which case you are in, judge accordingly.
Personally: I never do take homes, unless I have a good feel for the company. If the take home is given by HR. Welcome to my circular file. This is a hard and fast rule at this point. If the hiring manager wants me to do it... That's different. But I want to talk to them and decide if this is worth my time first.
The old game of chicken: https://en.wikipedia.org/wiki/Chicken_(game)
As usual, the party with the best alternatives wins.
It would probably take me an hour to set it up again.
Still I produce quality Python code (almost) every day. To me, the skills are entirely orthogonal.
But I was commenting on this:
> If you can't setup a python virtual env in about 30s... don't apply for a python job.
Oh and I should also add that I've never had a recruiter tell the truth about how long one of these should take. They always lie with some ridiculous number of hours that I know only a tiny percentage of the entire programming population could perform at.
I agree, yet there's a bunch of people in this comment section claiming you should give people take-home tests first which makes no sense to me.
Instead of spinning your wheels and wasting your time on your own.
That's what I'd want an employee to do.
I care less that you are having issues than I do about a complete shut down or lack of communication.
Update: Already getting downvoted for this response and not even a comment to explain why... classic.
The risk is, by asking, you indicate you don't know how to use standard industry tooling.
"hey, your starter code is not running in my environment, have any documentation about how to set it up" is a better question to ask when it's some custom script they hacked together.
But if that's actually the problem, then it's in the best interest of both the employer and the applicant that they not be hired. The applicant isn't ready, and is likely to have a miserable time on the job.
This kind of wordplay makes sense from HR's perspective. Ex: it "wouldn't be a fit for you anyway", there are "other jobs where you'll be able to grow more", etc.
The best interest of the applicant is always to get an offer (or stop the interview process of their own volition). Not get rejected by a screen and get zero feedback. Obviously this costs more for the business, which is why they have incentive to use that kind of reasoning.
Even if you shouldn't take it, competing offers can be used in various negotiations. And getting practice at interviewing is how you get better at interviewing (and at your actual job).
I don't think this is a controversial statement. If such an applicant gets the job, both the applicant and the employer will be unhappy and someone will either have to quit or be fired, which also sucks for all concerned.
> If such an applicant gets a job...
Getting the job offer would not cause any of those problems. By getting it you would build valuable interviewing experience. Taking it is the thing that's not in your interest. That's all I'm trying to say.
I don't give tests, I don't give technical interviews. I scout amazing developers and try to convince them to join my team. Not only do you get top notch people like that (who very much appreciate not being "tested"), you also save a ton of time.
Your job as a manager is to proactively search for the best people possible to join your team, hire them and keep them as happy as humanly possible.
The number of developers who have something recent and meaningful on GitHub is extremely small.
I’ve interviewed and hired a lot of great developers. Very few of them had much of anything on GitHub. Often the most I could find were one-line patches to projects somewhere.
If you limit your hiring to GitHub OSS superstars, you’re going to miss out on a massive number of excellent developers.
Extra bonus points for hiring the maintainers of libraries/software you're actually using.
I don't think it's an issue that some companies want to use GitHub as a shortcut to vetting developers. It does limit the pool they're fishing in, but that's what all vetting processes do.
There's an order of magnitude more qualified developers who don't have the time/aptitude/patience for contributing to OSS. You're shutting yourself off from the bulk of the hiring market by limiting candidates to OSS contributors.
I have seen software engineers getting lost trying to solve a trivial problem because they like to always write code that scales well or showcase their knowledge of software design.
Take home projects are great to learn about a candidate's decision making process given the use case and constraints of the final solution.
IMO that would be hard to assess by looking at a repository.
As far as having an entire team of top performers, IMO most companies don't need that.
Where I work most of the stuff we do never gets released as a final product, we may work on a new feature for six months and then gets killed, because the business requirement changed, because we partnered with another company that already provides that functionality or because our VP changed his vision.
Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them.
We do have a few way above average coders leading the way, but most of our team is made of good coders, not stars.
So far that's working great for us.
Starting a project from scratch, shipping it and supporting it is quite a bit more difficult than regular company drudgery. I've never had an issue with someone hired via this method struggling to perform once in the door.
> Having a team of just top performers would not change that fact, but you will still have to pay top salaries and get into bidding wars with other companies trying to poach them.
You should always pay a fair salary, but you'd be surprised that when you use this method you can hire some amazing people for "regular" pay. Respect goes a long way with top notch developers. A lot of the best people won't even do the leetcode dog and pony show, even if it results in very high pay. People will usually compensate for what they view as a toxic environment by demanding a higher salary (and quitting a couple of years in).
Beyond that I scout for people who launch projects from scratch (large or small) as it's a great way to see how people solve problems and how good their product sense is. Sometimes I'll just browse through the top charts on GH in the language we're using. Between GH, Twitter, HN and Reddit I see dozens of projects a day (more than I have time to investigate).
One nice thing about doing this is it tends to snowball. Once you hire a high profile maintainer or several OSS enthusiasts, it makes it much easier to hire more. OSS developers will typically be skeptical of "business people", so it's best to either have an OSS presence as a manager or reach out through another OSS dev.
I also stress that people can keep working on OSS in their free time (and keep ownership over their personal code). I go into the first meeting prepared to offer the person a job and also offer them the opportunity to interview anyone else at the company they may want to talk to.
The results have been very positive, even if they don't take the job. They're usually flattered that you've done the research on their work and value it enough to offer them a job sight unseen. I would estimate that I have ~75% acceptance rate.
Similarly for developers where software development is their job, they may simply not feel like continuing to work in their non-work time (or they may have a family they would like to spend time with, etc).
But similarly take home tests aren’t going to give you anything interviews aren’t going to show - if you’re concerned about people lying about their competence you should assume that they’re able to pay someone to do the work for them anyway. After all the threat model for a take home test is dishonesty.
I really think a lot of people think the goal of an interview is just to determine “can this person do the mechanical part of writing code” which is the easy bit.
- it doesn't value equally candidate's time and company's time. The candidate has to spend hours/days to do the test, while the company evaluates the result in minutes (1h at most)
- they are always open-ended, so you can always do more... that means the more hours you put on it the higher the chance of passing. So, if I spend 4h, but you spend 10h, all things equal, you got the job
- if they are underspecified and you dare to ask questions... be ready for a huge email-thread coming and long waiting times (in the order of hours instead of in the order of seconds/minutes as it would be the case if you were doing the test "online" with them)
- if you ask questions: how many questions is good enough? Maybe I'm asking too many? Or too few? I cannot really tell by the way they write their email answers to me (I could tell if I were asking the questions in a video-call, though... but damn it, this is a home-take test!)
Take-home tasks are the wrong solution to this difficult problem of our that is hiring.
I suppose that still selects for people who have 3 contiguous hours to spare. But it feels to me to be the fairest solution - it doesn't reward people who have endless time, and it doesn't subject poor candidates to open ended misery.
Sending the dev environment stuff before hand gives them time to make sure it's working first - and an opportunity to ask questions about the environment if anything is unclear. It definitely shouldn't take 2 hours to set up a development environment - if that's the case then it's either an inexperienced candidate or very poor instructions.
What if a candidate is way better but cannot spend 10 hours in an interview? Is it actually a good trait to spend three times the time allocated in bells and whistles?
As a company, you'll want to prevent this sort of scenario, but as the interviewee you really don't owe them anything so you can just say you're no longer interested.
If, for some reason, you feel compelled to improve their methods then you will have to succeed at their assessment in order to do so. It is a property of humans that to change a thing you have to be able to show you can handle a thing.
Even without taking my time to do the best job possible, it wasn't going to be a 2 hours job, but at least a 4 or 5 hours job.
It will never going to last as they say. For one reason, persons giving these tests are underestimating the time.
And if I said that I don't want the test, they wouldn't have hired me, which would be a shame. :)
At other interviews when I got to choose between live coding or sessions with questions and home assignments, I've always have chosen home assignments and always got an offer.
I am skeptical most people even attempt to do their own tests they create, I commonly see people suggest things will be 2-4 hours when they will be a day or two of work, https://www.bryanshalloway.com/2020/10/27/should-you-use-an-....
Yes, you can have your own engineers estimate how long it should take, but those engineers i) are already familiar with your problem domain and tooling, assuming the take-home test is reflective of those, ii) estimating is hard, even for small tasks, and iii) also don't have any incentive to name a high number vs. a low number, because a low number will make them look more competent
I have completed many take-home tests for jobs I wanted, and I don't think I have ever been honest about how long such a test took.
Politely say something like "After some reflection, I've decided not to pursue the position at [company]. Thank you very much for the consideration. I wish you the best of luck."
I would probably make an exception if they offered to pay me even a token per hour (or even donate the hourly to a charity - it's about incentives not money). I have made an exception when it was literally a 20 min code challenge for a promising lead.
On one hand, this does frequently stop things moving forward. On the other hand I never felt like I missed out.
One company I sent them my usual polite decline and got back an angry rant from the CEO! I responded with "Well, give me your product for free for 8 hours or so and I'll give you my time for free for the same length." He did not respond after that.
This is probably bad advice for very new engineers where a code sample can help differentiate you a lot, but I wish more senior+ engineers would refuse these dumb tests.
The handful of times I've dumped a bunch of time and effort into an at-home screening assignment, it took far longer to do well than whatever the allotted time was made out to be. Emphasis on the "do well" part.
My (unscientific) sense is that these things are really just screening for people willing to jump through unreasonable hoops.
Counterpoint: what idea can be had from rushed / time-constrained / deliberately careless work examples?
Any junior can copy / paste some functional yet poorly done code. Or make ChatGPT come up with something, I suppose, now that that's a thing.
Experiences vary, but every job I've taken and liked - no coding tests of any kind. More of a discussion about an example piece of code, for that part of the interview, but no requirement to come up with new code.
In my case, I actually do want to know how quickly people can pick up a new problem and get started writing some code, and that's how I evaluate what they deliver.
> Any junior can copy / paste some functional yet poorly done code. Or make ChatGPT come up with something, I suppose, now that that's a thing.
I have two thoughts about this.
1. If people can deliver, they can deliver.
2. If you come up with an original problem (rather than lazily copy and pasting some fizz-buzz type problem), you'd be surprised how many people fail to apply any basic problem solving.
> a discussion about an example piece of code
I do find that it is useful to keep code exercises short, but ask a few questions about their solution as a follow up. People often struggle to explain their solution if they just copy/pasted.
The best way for senior folks to get a new job is through referrals.
At what point did programming exercises during interviews for programming roles become anything more than a litmus test to weed out fraudsters early in the process? It's not supposed to be an unpaid hazing ritual, don't turn it into one.
We're talking about technical engineering positions here folks. Once you've filtered out the frauds have a technical deep-dive conversation with the person that is germane to the role and their CV.
You need to vet their communication and collaboration abilities and fit anyways. A take-home exercise doesn't produce any of that information. It's _deep_ in diminishing returns land vs. having them spit out a toy program from blank slate -> executable in a few minutes. If they struggle with that, you likely have a fraud, end the process, NEXT.
There's a reason a month would pass before benefits kick in; it's enough time to see if they can do the actual job and get along with everyone.
And yes, this is abuse. Demanding meaningless work, for free, for the chance to get hired? Because you can't figure out how to interview in a way that allows you to judge competence? Yeah, I think it's fair to call that abuse. At least exploitation.
In a remote first, or at least remote friendly, world, take home tests seem inevitable.
If I take a day off and come in for an interview, it's going to take them a day of their time, too. They're not going to waste my time, because it's their time, too.
But for a take home, it costs my time, but not theirs. They have little incentive to be considerate of my time, because they don't feel the pain.
> "The test was as much about seeing if the if the candidate could deal with manually setting up the build environment without assistance as much as it was about the coding problem itself. If the candidate couldn't get the tools installed on his own or felt insulted it wasn't his ideal IDE, then they figured the candidate would not do well as an embedded engineer on more complicated boards with even worse tool chains."
In general however a take-home test voids the primary rationale for the real-time on-site coding interview, which is that it's very easy these days to complete such tests without knowing much about the material via various routes, as academic CS programs continue to demonstrate. It's also true that academic CS programs tend to be terrible when it comes to learning about tools, toolchains, build systems, virtual environments, etc. There are some efforts to change this at least:
https://missing.csail.mit.edu/
Even that's not that comprehensive. Searching "introduction to toolchains and complex build systems" reveals how difficult the issue is. It's the kind of thing that should probably be included in the job description.
It is cruel, but it is numbers game. The take-home is easier for hiring managers given the macro environment and would help to cull out false positives (hence "numbers game"). Our industry has way passed the point of not playing these numbers game.
The most friendly ones I've seen is a "take-home" done onsite with generous time given, and an engineer can answer questions. It helps to simulate work environment as much as possible. The upfront cost is extreme for the companies though: you put up your engineers time for these onsites. You did end up with good hires, but the time limit is real (v.s. take-home), and that shoot up your false negatives through the roof. But it does respect candidate's time, and friendly.
For companies can afford (in the sense that good candidates just walk away), "take-home" portion also helps "alignment". If you spend extra time on it and really want to get in, you are probably ideal (or desperate, but still ideal, gives the company more leverage). These are given no cheating is happening though.
I have to admit that I never did good on these "take-home" tests. Setting up environment normally was not an issue, just without an engineer to clarify constraints, I often end up with less-than-commonly-expected solutions and make the interviewer hard to judge the end-result (most "take-home" from interviewers' perspective are very structured like exams, you get x points here, y points there, if you go off the expected path, it was hard to score these points). ChatGPT probably will do a better job than me at this these days though.
Perfectly reasonable task, IMO; it's even relevant to the gaming domain; you could do a simple version in an hour; I spent about 2 hours as I wrote a test harness and let it run overnight keeping score of how many maps it succeeded on and how many it failed on.
The interviewer used my submission in the on-site as well to ask a few questions about it.
Your numbers game comment is spot-on. From the employers' end, the hiring market problem looks like:
There's high pay for software engineering roles.
There's no effective credential mechanism. (I think this good, but it's not helping employers here.)
Because of the combination of the first two, lots of people are motivated to try to land a job.
Because of that, lots of people who aren't really qualified enter the pipeline.
Because they're not really qualified, they keep applying over and over, while the qualified eventually get hired.
Because of that, the applicant pool has a massive adverse selection problem (the average applicant is *much worse* than the median employee) and companies have to find a way to filter out the worst-of-the-worst before applying significant manual effort.I think if I was to refuse take-home task this would be the exact official reason: Because I believe take-home tasks are unfair and incentivise candidates to waste as much time as possible for no benefit for the company.
Instead, I do pair programming session debugging a piece of code and unit tests. It is not about the solution, it is about getting to know the candidate as he/she is working on a task.
When I was looking for a job in 2020-2021, I think it happened everywhere I got past the original opening interview round.
Sometimes the task was trivial or even submitting code samples (which can be problematic if the relevant examples were done for a previous proprietary interest that would not be agreeable to share this code). Other cases were more elaborate, requiring a half-day of work or more.
In one instance, I had an assignment for a front end JS / UX page, completed the assignment then heard back from the prospective employer that according to their review of my "homework" submission, I was overqualified for the position.
I mean, you can say "no", but then in all probability the prospective employer is going to proceed to the next candidate.
If it’s not substantial enough to be interesting you can just stick to standard interviews which also mitigates the risk of cheating.
Generally [0], people want to hire good candidates, not shit on people who aren't yet ready. Asking questions doesn't mean you're not ready. Not asking means you definitely are not ready.
[0] Yes, there are some assholes out there that love to torture candidates. Seriously.
Try giving a bit more than asked and explain your thought process clearly.
If you can't come up with a full solution at least mention what you would like to try if you had more time.
ALWAYS write in a positive tone, try avoiding words like "can't" and "difficult".
Provide a short summary about what you learned with the test.
I got offers this way despite being late for several hours or even days.
Unsure how anyone who can't do take home tests is qualified to do the underlying work.
Make room for those of us who THRIVE on take home tests and struggle at whiteboard interviews, I'll be happy to take that job.
Some people work better one way or the other. It sets them up for success rather than making a take home project an arbitrary filter.
And I hope the same take home project is used for every candidate and it's not a scam to get work done for free.
I've had two jobs which started with take-home tests. Honestly, I found them fun - they were maybe on the level of a 2nd or 3rd year undergrad CS homework assignment.
The jobs also were the ones I stayed around for the longest.
Also, does it have business value? If so, it gets more suspicious.
If a company is asking you to use your free time before you're hired, that's a red flag for their expectations if you do end up being hired.
Seems net worse for everyone.
The sdk was broken!
So I produced a minimum viable repro, wrote up a bug report and even included an analysis of where the bug was.
The company had the gall to still ask me to do a take home using the patched sdk!
If your take home test is broken, stupid or not trivial to setup, it's not the applicant who is at fault.
And the requirements might be reasonable for an already setup project, but when you have to do it from zero; unless you're scaffolding new symfony projects every time a new release is out, you're going to waste at least an hour scaffolding/googling/installing/wiring up all the dependencies/auth/orm you might need for you flow. It's not as bad as within the NodeJS ecosystem, but all libraries/frameworks that have a yearly major release, like to break things along the way so you're never going to hit the ground running in a couple of minutes.