Also curious to hear about any bad ones you've done.
The worst one was about 15 years ago. The task was to design and implement an ML system to solve a complex network optimization problem. It was so complex that I was given three months to do it. My estimation was that I could do it in three months if I worked on it full time, which was clearly a ridiculous ask.
Lol! That's quite ridiculous.
The interviewer rejected it and said he wanted working and tested code, which I estimated would take nearly 100 hours to do.
I told him I wasn't interested in working for free, but I would gladly do it for my normal consulting fee that I was charging at the time.
Didn't hear back from him. Dodged a bullet there.
I have probably done about 6 take home assignments in my 14 year career and 5 resulted in a job offer. In a recent company I worked for I was the hiring manager and we had about a 40% assignment completed to offer ratio. Which seems like a fairly decent conversion ratio, from the average candidates side that would be close to 1 offer for every 2 assignments.
Is there a correlation with people hating on them because the ratio is skewed much less favorably? Do I have too subjective a view? I know there are companies that give an assignment as practically the first step in the process; but I would never engage with that since the risk/reward is weighted entirely in the company's favor (which under the surface, is an even bigger red flag).
My favorite take homes, and almost always what I have come across, are the "build a mini api" ones. I will typically just draw from the structure I have in place in other applications, implement the custom functionality, and do a readme that delves into the philosophy (in a light touch way) of the choices I make. Then the conversation of going through the implementation in a technical interview is generally a relatively relaxed process.
If a potential employer wants to evaluate my ability and style for writing code I have a GitHub profile with numerous projects populated over numerous years. If that is too much trouble then why the fuck would I do a take home assignment for them?
- Recently: Given some basic intermediate output from LLM system, build out evaluation/quality tools, especially for dealing with hallucination - ~8 years ago: Given a database table of restaurants, identify near-duplicates to merge
On the hiring manager side of things, my candidates generally enjoyed doing a take-home to build a ML model for the Li & Roth question classification dataset. That involved a bit more creativity and it was very role-related.
After being on both sides of the interview process, my current feeling is that take-homes are a decent fit for junior roles but take-homes are not a good use of time (on either side) for senior roles.
It was up to you what you wanted to do / how / if you wanted to deploy. They only asked that you write how you would've liked to receive such service from another org.
Then in the day of the in person interview we reviewed my code / README / tests ; was very nice.
That speeds things up no end.
If it isn't worth four or eight or N hours of your staff's time to observe the candidate working on the problem, then it is probably not worth four or eight or N hours of the candidate's time.
Because what are you trying to measure?
How well the candidate lies about how long the project took?
How willing the candidate is to work many hours?
How much free time the candidate has to dedicate to their job search?
How well they GGS (Google and GPT and StackOverflow)?
How resourceful they are at hiring a freelancer?
Etc.
Sure those types of behaviors often resemble people's actual jobs, so maybe a take home coding task measures what your company is really looking for.
But realistically, inexperience and desperation are the most likely reasons someone will be excited about your take home challenge. The best skilled and most experienced candidates -- the kind people who know them want to work with -- probably won't be excited. Good luck.
> But realistically, inexperience and desperation are the most likely reasons someone will be excited about your take home challenge
Also, why they'll become more common - used by wolves in sheep's clothing. Selecting for those who are more junior or simply malleable.
Hence my fairly strong stance; we're interviewing 'together' (same time, remote/local/whatever)
At your stage (startup with 2 people), you are much better focusing with designing your product than designing an interview process. Hire someone you know or freelancers.
Architecture level tests are a sweet spot for nos. AI is not good at this, but senior candidates are often very good at them. You can give them template architecture to start with or work on, something that AI is known to trip over. The only way to know what AI is tripping on is to use it.
If the candidate is better at using AI to write code than the team, bring them in!
If a candidate uses AI as a help to solve their task, yet the output is still good, great! It's their job to solve the problem, with whatever tools they have.
What are some of the things you've noticed ChatGPT get wrong vis-a-vis schema design?
I would probably skip a take-home for candidates at all for now. You probably have some specific pain points already. Try to find experienced TypeScript developers with some experience similar to your pain points and recommended to you by people in your network.
If you made me create a take-home assignment, I might make it rather open-ended and say do a compare/contrast of the stack-auth REST API versus one of the other options out there using any programming language and be prepared to describe code in both examples and strengths/weaknesses? This will help you maybe identify developers who self-select on "I'm interested in the stack-auth problem space . . . " Plus, that feels like a reasonably-sized effort for a candidate to me and could maybe be fun.
If you had a technical component during actual interviews, I think it is totally ok to ask someone to code up a small, simple Typescript example function of limited scope. Frankly, that's probably enough for you to get "yes/no" hiring signal.
I'm, err, older than you and will gently ask that you don't focus entirely on people who graduated since you did.
If the culture is take home, please don't convert it into a live interview; you'll get neither.
If I can't make a case live, but must go off with some pet project while imagining their behavior... I don't want that interview/work. Folks will certainly disagree, just here to remind of agency.
The best firms gave options. Am adult, will work under agreeable terms. One of those terms is "together" (time, not space), despite my own social anxiety. The same way the business hopes to select with this process, I will too.
None of this is to judge others - do what you prefer, but please do realize this 'take home' approach can be taken advantage of [again, for either side].
I get that some people want to be respected for their time, and that has to be built into the process before and after any take home tests are given - compensation, trial buddy, flight & hotel room, detailed feedback after the test.
We're also doing a paid trial period, though that would be after the interview + take home.
If nothing else, take this as a scar some of those who are interested may be carrying
In the sense of would this be a problem that might reasonably come up in the course of working elsewhere ... and how comprised would you be had it done so given you'd seen it before and signed right to your solution elsewhere.
Fast forward a decade and several jobs later, my favorite take-home so far is simply a no-nonsense assignment that reflected the actual work I would be doing. As a frontend person, that meant making a dashboard in React or whatever JS flavor I wanted, with whatever tools I wanted, for a co-review a few days later.
They asked me to make something like a movie browser, kinda like Netflix, using a provided list of 100 or so movie titles. It was a pretty simple project that only took a day or so, but showed enough basic React listing/filtering/searching/state/context usage, and some simple API lookups (used it to get cover/poster art and some metadata from IMDB). It wasn't a particularly challenging assignment, but it wasn't meant to be – it was just a normal frontend job at a traditional (non-tech) Fortune 1000 company.
After I submitted the assignment, the interviewer spent an hour or so walking through it with me, asking questions about certain parts and offering constructive feedback on others. It was a really good use of both of our time: them to see how I think through user-facing problems, and me to see how they like to manage work and communication.
I got the job, and did very well in it, and helped solve a few interesting challenges at that company (the dashboard was complex enough that it had interesting UX, IA, error handling, user flows, visualizations, etc.)
---------------
One take-home I did NOT really like was from an online interactive mapping company I really respect. They asked me to make an animated analog stopwatch, similar to the kind you'd hold in your hands at a race. I'd never really done any animation before, and their product didn't seem to have much of it, but that was the assignment.
I made one and it worked, with some caveats. The assignment didn't have much detail to begin with, so I submitted what I had and told them I'd be happy to modify it or rewrite the whole thing to spec, in whatever stack they wanted. A few hours later they told me sorry, but no thanks, and that was that. No further feedback.
It annoyed me because the take-home didn't really seem to reflect any sort of work I'd actually be doing anyway (web dev + mapping), which I actually had a moderate amount of prior experience with. The GIS (geo information systems) world has plenty of its own nuances and interesting challenges, but none of that was tested or discussed at all. To this day I still don't know why they wanted an animation challenge for a web mapping app.
Presumably they cared more about the graphics side of things, or it was just a chance for the right candidate to write something really cool from the ground up to impress. The assignment itself was flexible enough that it could've been anything from "here's a npm lib that does this with a SVG" to "I wrote an entire 3D renderer with ray-tracing to better simulate the watch face". I did not impress :)
----------------
Oh, and one more bad/ridiculous experience: One company's first "take-home" wasn't even a coding challenge, but some sort of convoluted personality test. They made you sign up for a third party platform to take a specific personality test that analyzed your personal and professional habits, etc.: https://www.assessmentday.com/ccat-criteria.htm
I laughed in their face and walked away. Guess that shows my personality, eh? Bullet dodged.
If not, are you going to miss a lot of people who have a no-take-home-problem policy, or are skeptical of take-homes that "resemble" work, really being someone else's work?
But you do bring up an interesting point.
> If they "resemble" work, shouldn't you pay applicants? If not, are you going to miss a lot of people who have a no-take-home-problem policy, or are skeptical of take-homes that "resemble" work, really being someone else's work?
I think it's kind of a two-way street and self-selecting group, isn't it? People who don't like that sort of thing filter themselves out, and companies who want mid-tier candidates have an easy way to filter out both people with zero skill and people who are experienced enough they don't have to play games.
If you're the kind who can get work with a no take-homes policy, you probably don't need to be applying for jobs out of the blue like this anyway, rather than just working your connections.
Otherwise, as a candidate, I vastly prefer a reasonable take-home over the usual inane interview hoops (talk to four people in HR, a few managers from different departments, the dept head who you'll never work with, etc.). And arguably it SHOULD be like the real work, both for the company's and the candidate's sake, to see if the day-to-day would be a good match.
And if the company happens to use some of my code without hiring me? Okay, well, so what? I spent a few hours of my time, made some random corner of the internet a bit better. I got to practice a little, they get a bit of free work and I get a little cheated, ok, but life goes on? More productive than playing video games, at least, lol.