If even it happens some 10 percent of the time, it'll get a reputation for being unreliable and would lead to "oh, why waste time, I better ask the candidate directly" situations.
But other than that, it's a very creative idea. Really. 10 out of 10.
EDIT: Idea is great nevertheless.
This is partly why I show the source Snippets (Q/A Pairs written directly by the owner) below the summary as a way to verify the information. Kind of like the AI 'showing it's work. It also let's see more about the owner which is a nice side benefit, or maybe the main benefit.
I can also turn off the AI summary part and leave the AI search part. If this becomes a bigger thing, I might give users a way to enable/disable the potentially hallucinogenic part, but it'll be their choice.
I think there's a lot of work to do to work out the kinks with things like hallucinations, but I think we forget sometimes so much of what we do is statistical in nature already. Your car has a statistical chance of breaking down within x timeframe and while you're on the highway, it's just, low enough that you don't worry about it. I think AI will be a similar thing where we have to get comfortable with how we evaluate and mitigate risks, but like many things, they'll never be 0.
Additionally you don't need ML to get your computer to show you confidently something that is completely wrong, all it takes is to multiply specific floating point numbers repeatedly, really.
Just dry run through the scenarios and assume LLM's output is wrong even if 10% of the time.