Hundreds of candidates? That's significant if not an exaggeration. What are the stark differences you have seen? Did you inquire about the candidate's use of language models?
Candidates who use LLMs generally get through 4 or 5 steps in the interview question. Candidates who don't are usually still on step 2 by the end of the interview (with rare exceptions), without their code quality being significantly better.
(I end up in 1:1 interviews with perhaps 10-15% of candidates who take round 1).
I suppose, if you are specifically looking for coders to perform routine tasks, then you'll get what you need.
Of course, you could argue that ~90% of a programmer's work day is performing standard tasks, and even brilliant programmers who don't use LLMs will lose so much productivity that they are not worth hiring... Counterpoint: IMO, the amount of code you bash out in a given time bears no relation to your usefulness as a programmer. In fact, producing lots of code is often a problem.
I think it's a reasonably balanced interview process. Take home tests are useless now that LLMs exist. Code interviews are very time consuming on the hiring side. I'm a firm believer that hiring without some sort of evaluation of practical competence is a very bad idea - as is often discussed on here, the fizzbuzz problem is real.
Because it seems like tougher real-world technical problems (where there are tons of dependencies with other systems in addition to technical and business requirements) needs for the dev to have an understanding of how things work, and if you rely on an LLM, you may not gain enough of an understanding of what's going on to solve problems like this...
... Although, I could see how devs that are more focused on application development and knowing the business domain is their key skill, wouldn't need to have as strong an understanding of the technical (no judgement here, have been in this role myself at times).
Yes, definitely, though I lean more on the 1:1 interviews for that. I understand the resistance to this from developers, but there's a lot of repetition across the industry in product engineering, and so of course it can be significantly optimized with automation. But, you still need good human brains to make most architectural decisions, and to develop IP.
That seems like exactly what the person you're replying to is saying - that sounds like basic standard product-engineering stuff, but simpler, like any of a million examples out there that an LLM has seen a million times. "Here's a problem LLMs are good at, wow, the people using the LLMs do best at it." Tautolgy.
So it's great for finding people who can use an LLM to do tiny product things.
In the same way takehomes had all the same limitations. More power to you if those are the people you are looking for, though.
But it also sounds like a process that most people with better options are gonna pass on most of the time. (Also like with takehomes.)
Yes, product engineering, the thing that 90% of developers do most of their time.
But if you want to create a paradigm shift, you need to do something new. Something LLMs don't yet know about.
That's a mistake. There are plenty of people who are not good multitaskers and cannot effectively think AND talk at the same time, myself included.
IMHO, for programming, monotaskers tend to do better than multitaskers.
Each to their own and maybe their method works out, but it does seem whack.
This is why I wouldn't hire a person who doesn't know how to do this.
Edit: there are also recruitment agencies with ex-engineers that do coding interviews, too.
Sorry this is a little harsh, but how do you get anywhere near 100 people before realizing the approach must be horribly flawed, and devising and implementing a better one? Surely it behooves you to not waste your employer's time, your own time, and the time of all those people you're interviewing (mostly pointlessly).