And then: I want my point to stand: for a given role, there are good hires and there are bad hires (where "bad" includes "meh"). There is not a lot of value in grading past that. But lots of hiring processes are premised on doing exactly that; they take candidates who can demonstrably perform the task required for the role and then beat them up with trivia questions, with puzzles, with nerd dominance quizzes, and with subjective stuff.
I point that out because if you can find a way to accept that, you can streamline a lot of recruiting process and make interviews less painful for everyone. The way I finally managed to make that leap of faith was to reflect back on everyone I've had a hand in hiring that didn't work out, and then consider the process that hired them. Then I think back on everyone I've hired who has surprised me with their awesomeness (some of them are even people I'd negged). I am now sold on the fact that most tech interviews are voodoo rituals.
What I would do/am doing instead:
* Get people out of HR/phone screening as quickly as possible.
* Have an explicit up-front call with me or someone senior in the hiring process to level-set people, try to defuse some of the tension, explain the process, and answer questions. Things go faster and more smoothly when candidates are convinced they want the role. The amount of sales effort it takes to get a candidate into the pipeline is not enough; you need to engage them on what the job is like, let them ask what the day-to-day is, and get them revved up before they interview.
* Minimize phone screening, both in length and in depth. I am on a mission to get rid of them altogether but am not convinced I'll succeed.
In the interim, what I'd like to do is take all the effort and expense we'd sink into hour-long phone screens and redirect it to coming up with a small rotating battery of really, really good questions that we standardize on.
* Replace phone screens with challenge problems. Calibrate them for the length of time a series of elaborate phone screens would take. Try to make them fun. Here I admit I'm not sure what I'd do back in my pure dev jobs to make this work, but I'm sure I could come up with something.
* Give the on-site interview team a full dossier of objective facts about what our up-front process learned about the candidate, so that nobody is trying to read tea leaves out of resumes. Candidly: I barely glance at resumes anymore, and I wince when I read interview reports that go "I read XXX on the candidate's resume and so asked YYY".
The most important thing I think I've learned about recruiting in the last year --- apart from getting better at getting people in the door --- is to generate sets of facts that are easy to compare between candidates. I'm learning a lot by reviewing the history of how people do on challenges, and I really very much want to be in the same place by the end of the year with all person-to-person screening interviews. A nice side effect I'm hoping for is for prep time on phone screens to approach zero, and for actual phone time for interviews to approach ~20-30 minutes. We'll see.