If they have a proven track record of figuring things out experience is minimally valuable.
I can't understand this point of view at all.
When Google wanted to build the world's best JS JIT they didn't hire anyone who was generally able to figure things out - they hired the person with the most experience in building dynamic language JITs in the world. Experience is everything! Experience is knowing which rabbit holes to not go down, knowing who to speak to when you need help, what ideas haven't been tried yet, etc.
And then there is the issue of actually retaining top talent...
I think the issue we have nowadays is that most software companies are creating mediocre to shitty software that solves no real problem, but selling themselves as the next Google or Apple. Most engineers can see through the bullshit and thus don't feel connected to the product. Then it just becomes a contest for who can realistically write the code for the shitty product for the least pay.
If you have to for example build a new operating system kernel, or a new compiler, or implement a machine learning system, hiring a bunch of well-intentioned smart people with the vague idea that they can probably figure it out as they go is a bad idea. You sometimes need someone with a track record of being able to do the work you need done.
I've been interested in many fields, but can't afford to take time to establish myself in them unless there is also significant interest from a company looking to hire me.
and CISO of LinkedIn, Cory Scott: https://www.linkedin.com/pulse/evaluating-technical-talent-t...
This is called work sample testing and it works great.
You can do a work sample test for 'can this person add a view to a Rails app', but not for example for 'can this person build a team and develop a solution to a complex technical problem in a specific domain'.
In my experience, when you think you can't measure a thing in a work-sample setting, it means you haven't analyzed it. And, if you have no idea what that thing means, how could you possibly hope to interview for it?
For example: "can this person build a team and develop a solution to a complex technical solution" (that's several things, but OK, let's take a stab at it):
- Someone who can build a solution in a specific domain has to be able to analyze the problem into bite-size pieces. Given a complex problem description, write out tickets that need to get done. Can they identify sensible milestones, objectives and key results? Are the tickets roughly equally sized? Are the tickets self-contained? Do they come with an objective measure of "done"?
- Someone who can build a team to deliver a solution has to be able to estimate appropriate resourcing for a project. Given a complex problem description and a current set of resources, come up with a plan forward. Measurements include: a) Did they identify missing roles? b) Did they write clear job reqs with evaluation and hiring plans? A gazillion hiring managers couldn't write a clear job req to save their lives. c) Did they consider which positions may be effectively contracted out? d) Can they estimate what the P&L impact of this hiring plan is? Knowing how to read P&Ls and cost of development is a skill. e) Can they identify within their own plan which roles are really critical and which ones are nice-to-have? Being able to negotiate in the face of limited resources is a skill.
- Someone who can lead a team to deliver a solution to a complex problem has to be able to analyze when things go awry on a low-level. Put them in front of a PR where someone has subtly misunderstood a poorly-written ticket. Give them the context they need to understand why the PR is wrong. See how they tell people the PR is wrong, and how they react when people persist?
- Put them in a position where they have to talk to an employee who isn't doing well. Can they figure out how to be empathic while remaining professional? Did they say something that will get you sued? Opposite situation: put them in a room with a rockstar developer who's been harassing or badgering one of your employees. Can they speak to that person professional? Can they write an HR file note with a follow-up plan?
Folks have built highly specialized company teams with this and little or no interviews.