Ask HN: How do you hire outside of your area of expertise?
For example, you need to hire a machine learning expert, but you don't have any machine learning knowledge to test if they are any good.
Assuming you want to skip actually working with the prospect and you want to try to assess via an interview format, you're looking for one thing: Will this person help your company make money? I.E. will they get results.
While you can never answer that question for sure, looking for human beings that are results oriented is a reasonable first step. Machine Learning as an example is typically a project based type of work. Ask about their projects, how it went, and what was the result. If you talk to enough people about the results of their work, someone will stand out (maybe many people)... then you can move along to what normal hiring managers do anyway: hire who they like the most or pick out of a hat.
The other thing you can do is get expertise in the area.
"Results" is a funny thing to define though. For example say I know nothing about coding, and I hire someone to make a website. Based on their "results" they make good looking fast websites without bugs. I hire them. I get my site. I need a change and suddenly they are not available or got expensive. I ask someone else to do it and the say "maaan, wtf they have used Perl and a weird JS library for the front end, and purescript and I don't know how to maintain that. I'll need to rewrite it".
With machine learning, to evaluate the results, I think you'd need to know a bit about machine learning and what P(good work by this person | company made money from the model) is vs. P(null | company made money from the model)
This seems absurd at first, but it's a good idea in general. If my work involves hiring someone who does X, I should definitely have enough knowledge of X to make decisions on that. It doesn't have to be enough knowledge to implement X myself... but enough to make informed choices.
Try to flip the roles. Treat the applicant like a consultant/mentor. Tell them about actual problems that you need solved. See how they solve them or improve on your solution. Compare between different applicants.
You'll want to look for clarity in their solution. Someone who overcomplicates their solution is probably bullshitting you. Someone who slows down and explains everything clearly probably understands what they're talking about. Like someone who says, "I would implement ABC here" and doesn't explain the advantages and disadvantages of ABC likely has weak knowledge.
With some of the smarter people I've worked with, they make no assumptions, and establish terms. "Are you familiar with ABC? No? Well, you know A? A is flawed for this solution because ___, and so we use ABC instead of A, even though ABC has this disadvantage over A."