No wonder i didnt get the job, but rejection email said they thought I am a good engineer. I replied if they can tell me where I can improve. They did so politely.
Few months down the road, I improved and applied again. I got the job
Even with the most perfectly defined job description, you might have many candidates who "fit" that description, so you have to weigh a lot of other factors which run a huge gamut. Many, many times I've had multiple candidates that are more-or-less the same in terms of experience, technical competence, ability to learn, get shit done, etc - so you start looking at other factors and it can get down to things like communication, personal hygiene, general awkwardness, etc.
It’s entirely possible that the candidate was just off their game that day, or they weren’t “warmed up”. I get noticeably better at interviewing after the first few. It happens, but the interviewer has the information they got, and it’s not possible to totally control for that kind of variance.
What hiring managers should be actively minimizing, that most hiring managers I’ve seen do not, is variance that their interview process introduces by, for example, not standardizing and training their interviewers.
If the interviewer says "I think you're a good engineer", the next sentence should be "we would like to make you an offer".
If I could hire people by reading their mind and understanding their ability, without requiring an ounce of preparation, I would do that.