I've never had an interview like that personally, but I have had interviews that were a bit more practical. Like instead of solving an algorithmic problem, it was: how would you lay out the database for a clone of popular website X, how would you handle stuff like pagination. Those are a bit less nervewracking than the typical "implement binary search in a chamber slowly filling with sand" interviews.
Great idea!
If I don't have a solution, I have no work from the company.
In fact it's the opposite since it favors nepotism and disadvantages people from historically disadvantaged backgrounds. This is the same reason CA introduced the recent law prohibiting employers from asking about salary history.
But what is a "reference"? Often, it's a free-form phone call from the hiring manager to a person you've previously worked with. I think there's a lot of room to improve LinkedIn's "recommendations" feature--they clearly de-prioritized it, as well as endorsements.
In a nutshell, I want my social proof to precede me in my job search.
Why do fizz buzz when you can get several of your friends to confirm "this person can do fizz buzz, and also these things that you are expressly hiring for."
1. People can lie in support of their friends, or be bad judges of ability. A project manager might vouch that this guy is godlike when the reality is that they've just been working late whenever the boss is around and hacks everything together clumsily.
2. Some people don't want their best employees to leave, so they won't want to give a strong recommendation to someone, especially if that person is currently employed.
3. Whatever circumstances that causes one to leave a job might not put them on the best terms.
I feel that recommendations are sometimes like asking someone's ex about their character.
People who have been lucky enough to work great mentors, and smart & supportive colleagues, are at an advantage.
I personally didn't find one of these environments until 5 years into my career, so I'm well aware of the closed minded, short fused, agitated, demotivated, and political developers that exist out there. I know this may be hard to understand for anyone working in Silicon Valley where smart and open minded colleagues are probably the norm, but the other perspective is worth keeping in mind when it comes to references.
If a third-party recruiter is involved, they will get in contact with a developer that they deem to be sellable. Once they've got his/her details, they'll pass them on to an employer, who will then interview them for the role.
The flaws with this approach are:
1. The recruiter doesn't know if you're any good or not. They probably don't care, but a good developer can be sold for a high commission.
2. The interviewee has to interview when it is convenient for the employer
3. The employer has to run a ton of interviews to find a viable candidate, and (probably) come up with a proper interview process to replace their crappy one.
If, instead of having a skills-based interview with the employer, you had one with the recruiter, who would only take you on as a client if you matches a certain level of skill/talent, in theory everyone's problems are solved.
Recruiter: Adds real value to the developer, can sell a provable talent to a company quickly, and ideally with no worry about commission being lost by an employer rejecting an obvious dud.
Developer: Gets to "interview" whenever is convenient for them. Can be evaluated in multiple ways (take home test, face-to-face interview, etc), and can be given real technical feedback as the interview is not for employment, but to sign up for a service.
Employer: Can hold a shorter interview process, and get someone in quickly.
In terms of actual technical interviews, there are so many different ways of doing them that all I ever really want from an employer is some heads-up on what I'll be asked. Interviews are convenient at the best of times, so if I'm going to try and make time when I should be working I want to at least be prepared for the experience.
There are a few recruiters that try to approximate this; see a list here: https://github.com/vchernoy/dreamjob
A lot of these platforms are trying to solve the process for huge companies that offer whiteboard algorithm-based interviews, whereas a local agency down the road just wants a Rails developer with ecommerce experience, or a .NET developer with Umbraco experience. It's these companies that are more likely to want to use a recruiter, and it's tech bosses at these companies that are often wary of not only other peoples interview processes, but their own.
I recently experienced this process, when a recruiter had "found" a technical test, and progressing to the next round was dependent on passing this. Unfortunately it wasn't related to the role, I do iOS and the test was a bunch of SQL related questions. I have barely touched SQL anything more than a basic select statement since college 9 years ago, as all searches are done through fetch requests and predicates. Long story short, it's not relevant. The recruiter was arrogant and didn't want to entertain the idea of finding a more relevant test. These people should not be trusted playing a role in the interview process at all.
However, what do you mean by "the interview is not for employment, but to sign up for a service"? This is interesting and may be worth exploring.
My problem with recruiters is that they are salespeople. That's all they are. They take a CV, and they sell it to someone, and ultimately it doesn't matter if the person is good or bad, all that matters is that sale. As you've rightly said, they don't know enough about the technical side to provide a technical review.
So, why not have a separate team wholly dedicated to technical vetting? Have the process be completely open to employers, so anyone looking to hire a developer can see exactly what is to be asked of their prospective employees. If the questions are bad, the employer can say so and a qualified developer hired by the recruiter can tweak the process.
It's essentially adding a separate team to the recruitment process, one that builds a robust interviewing framework. They don't need to be full-time either. They can be working developers on a freelance basis that have gone through the interview process, and know what questions to ask based on the stack an employer is using.
For example, implementing a chat app.
As a general rule, if I'm asked to spend time on one of those coding assignments, I assume I have the job unless I really F it up. Only the top 2-3 contenders should be asked to spend more than half a day on one of these assignments, and they typically take half a day or more.
The best coding assignment I had from an interviewer is one where I got paid. It was part of their process, and wasn't a complete waste of time.
Things like "How would you create minesweeper on the iPhone" or "How would you make an app that's Airbnb for dogsitters?"
Just try to see how people would approach the question.
Not a fan of those...