It's infinitely hard. I mean, you take the same group of people and change the circumstances slightly and have a completely different outcome. Some groups don't gel. It's like last year's Super Bowl team, they're all still good but the circumstances have changed. Nobody knows how to predict who will step up in a situation and past performance is not a good indicator. The circumstances have changed.
Every interview technique has some horrible downside and therefore it's not really possible to correctly assess talent. Just biasing towards experience isn't solving the problem of figuring out who can do the job. Even just knowing who was capable would be a step up from the current state. Figuring out how to moneyball a team together to win the Super Bowl (hit launch goals) is a whole added level. Our 45 minute tell-us-about-your-previous-projects-interview is nowhere good enough.
My personal experience has been that this format 1) gives the engineer “home field advantage” by allowing them to go deep on a project they should know a lot about and hopefully show off a bit and be confident 2) allows us to really press on how deep the engineer actually understands the tools they use and what they do/did 3) allows us to understand if the candidate is a leader/get shit done orientation person or a passenger on projects 4) allows us to see how polished their technical writing is
The most successful candidates I've helped hire in my career were the ones where we had an informal process where we just talked about things and I went with my gut. They say that's not objective or measurable, full of bias, etc.
I toss my hands up in frustration.
so now it's a leetcode packets for everyone even though the candidates we've hired though them have been worse, our org is now filled people folks who studied for the test instead of having geniuine interest and unique aspirations
it's a charade since I had people from more varied backgrounds (economic, national .. etc) who were able to start decent carreres before having to change