In my position, I get the occasion to read a ton of assessments written by interviewers. Some of the most striking assessments are the ones where the interviewer is cock-sure that they completely nailed the candidate's utter and complete incompetence. It's usually an interviewer who's been asking the same question dozens of times over a year or more. They've seen every variation of performance on the question, and they've completely forgotten what it was like for them when they first encountered a question like that.
It's total lack of empathy at that point, and if the candidate doesn't exude near-perfect interviewing brilliance on that specific question, the interviewer judges them as essentially worthless. Interviewers like that sometimes even get snarky and rather unprofessional in their writeup, "Finally, time ran out, mercifully ending both my and the candidate's misery."
If I were to diagnose one of the causes of this phenomenon, I'd say it is bias. The interviewer best remembers the candidates who performed exceptionally well on their question, triggering the availability heuristic.
There are tactics that I think can be effective to bust those biases. One might be to put an upper limit on the number of times an interviewer is allowed to ask any given question. Once they've asked maybe 20 or 30 candidates the same question, it's spent. They have to move on to something substantially different.
There are some other experiments I'd like to run. One of them is to have interviewers go through a one-hour interview themselves for every 50 or so interviews they give. Maybe match up an interviewer who has a track record of being especially harsh on candidates for not giving a flawless performance on the question they've been asking for a while. The idea is to see if we can't bubble up some empathy.