As a method of finding good candidates it feels like it must suck but so few companies are willing to actually measure their hiring effectiveness (and be open about it).
As someone who has given hundreds (maybe thousands) and received dozens of these types of interviews, it's really not.
Generally speaking I'm looking for at least 2-3 of these abilities, depending on which question I'm asking:
1) Can you reason about data structures and algorithms when looking at a problem you haven't seen before? 2) Can you communicate your ideas effectively? 3) Can you integrate new information from a colleague while problem solving? 4) Can you write code that makes sense? Do you understand basic programming concepts? 5) Can you read your own code and reason about it?
There is at least two hidden attributes that I'm not testing for but do affect performance, so I try to account for them:
1) Are you comfortable with me, a relative stranger? 2) Can you do these things while dealing with a high pressure situation?
You will encounter high pressure situations at work but often interviews feel more high pressure (to some people) than most daily conversations about engineering.
That's it. If I can tell a candidate already knows "the right answer" to the problem, I'm usually disappointed because I'm more interested in watching them think.
It tells me they meet the baseline - they can learn well enough to at least be able to do this particular problem super quickly.
But it doesn't tell me any more than that, unlike the candidate who thinks for a pick, considers a few different approaches, writes something up, stops a few times to think about edge cases, satisfies themselves that they're covered, etc.
"Can you figure stuff out on the fly" is the test, not "do you have this memorized?" Fortunately, most people don't have everything memorized, so for now it's still a useful-enough filter.
The problem with alternative, past-experience-based types of questions is that most candidates are terrible at driving them. I would love someone to tell me an hour-long story of debugging something in a library, say - but usually, even with a lot of prompting, it's very hard to get much interesting stuff out of them.
So by putting some algorithmic questions on the panel too, at least they have a chance to show that they can code, even if they can't communicate much meaningful from their past experience.