These should not take 10 hours, unless you're learning Rails from scratch or something (some people do this). We've tried different ways of saying "don't spend more than 2 hours on this unless you really want to" and it doesn't always come through.
I don't know what fly's internal processes are, but I suspect the reviewer had to have submitted some bit of internal feedback on each submissions on why they're okaying it or not. A filtered version of it or even a separate candidate specific feedback would be just enough.
I've been at the hiring side of the table for many years and every time we share a handcrafted feedback to interviewees that we have passed on, we always heard back good things. They know we respected their time (and them), even if they don't agree with the particular feedback.
This feedback is not meant for them to get "better" or something. My point is, it could even be 2 hours the candidate spends, the least they deserve is knowing that it was actually looked at.
The submissions that we continue with check a bunch of boxes. The submissions that we pass on don't.
The problem is, we're not hiring people to build to a spec. We want to assess your decisions and ability to go from a basic problem to a first implementation. If we shared the rubric with folks, they'd focus entirely on trying to check the boxes and I don't think we'd get an accurate assessment.
Your point about valuing peoples' time is important, though. We have not yet found the right balance for everyone.
I wonder if your hiring process is skewed by the perfectionists who spend tens of hours of their submissions. Arguably that's the type of thing you guys would want to select for but it's not at all fair for those who timebox themselves to 2 hours or whatever.
I don't think you can spend more hours and get these criteria.
It's pretty rare IME to encounter interviews which are closely related to the actual work you'll be doing, mostly it seems like the main purpose is to filter out the heaps of bad/mediocre developers.
This sounds like a classic "my developer reckons he can do this in 2 hours (but never actually has)". The reality is your developer is crap at estimating.
A good developer will be lucky to produce 100 lines a day. if you doubt that check your git history (excluding autogenerated crap).
So unless the project you're setting people needs, on average, 25 lines, whatever you're setting people is clearly not a 2 hour project. And that's not even taking into account whatever hurdles you've inadvertently put in the task, strange build config, obscure libraries, out-of-date libraries, non-standard formatting, etc.
Could you tell us what the average length of a submisson is?
One task I got given a year or two back had an old version of Vue, linting rules that conflicted with the defaults of the 'usual' IDE you'd use, wanted you to create 3 new backend API endpoints and a new frontend page with non-trivial functionality. Plus they wanted units tests.
That's a 2 day job. They also claimed it would take an hour or two.
You could have told the candidate that, then. That would have been some useful information to have for the hours the candidate put into their submission. Or like "hey, we're going to just skim over this with a subjective list of check boxes".
I have enough experience that I probably would have smelled this before I got that deep into the process and noped out, but for the hopeful mid-level/new-senior devs this sounds pretty demoralizing.
If you ever get asked to do one of these exercises, it’s useful to try to determine where they are in this process. Ask about their hiring timeline, how long they have been interviewing for the position, and try to get a feel for how fatigued they are in general.
If it feels like they are just starting, ask how long you have to complete the exercise. If there isn’t a time limit, it’s better to wait as long as possible, so they can reject other candidates first and burn themselves out. If there is a time limit, ask to pause the interview process (insert some excuse here) before they send out the exercise.
If they are near the end and sound exhausted, that’s a good sign and the effort might not be wasted.
What detail is shared about what "checkboxes" the candidate met and didn't meet? From the parent post and your reply, it doesn't sound like much at all?
I've had a few bad experiences when sharing feedback with candidates myself and I would understand doing the checkbox approach for feedback and/or just never sending detailed feedback, but actually grading submissions pass/fail based on a subset of criteria you jealously guard from candidates essentially selects for lucky people. If I wanted to do that, I'd just shuffle the submissions by number of bytes and discard everything that's a multiple of 5 or something.
Did you try paying candidates for the home assessment? People's time costs money. Paying people not only helps attract candidates but also helps the company reduce the list of candidates to take the assessment.
I'm inclined to say the interview process is imbalanced with more power in the hands of the employer. If you're looking for the right balance, try turning over some power from the employer to the candidate.
The candidate's cost was 2-hours worth of time. Was the employer's cost 2-hours worth of time?