LLM inference outputs a list of probabilities for next token to select on each round. A majority of the time (especially when following semantic boilerplate like quoting an idiom or obeying a punctuation rule) one token is rated 10x or more likely than every other token combined, making that the obvious natural pick.
But every now and then the LLM will rate 2 or more tokens as close to equally valid options (such as asking it to "tell a story" and it gets to the hero's name.. who really cares which name is chosen? The important part is sticking to whatever you select!)
So for basically the same reason as D&D, the algorithm designers added a dice roll as tie-breaker stage to just pick one of the equally valid options in a manner every stakeholder can agree is fair and get on with life.
Since that's literally the only part of the algorithm where any randomness occurs aside from "unpredictable user at keyboard", and it can be easily altered to remove every trace of unpredictability (at the cost of only user-perceived stuffiness and lack of creativity.. and increased likelihood of falling into repetition loops when one chooses greedy sampling in particular to bypass it) I am at a loss why you would describe LLMs as "not deterministic".
"low" is the key word. If it's anything other than 0, it becomes non-deterministic.
If you use a temperature of 0, then the output of an LLM will be completely deterministic. Any given input would have the exact same output every time.
LLM's can make convincing arguments for almost anything. For something like this, what would be more useful is having it go through all of them individually and generate a _brief_ report about whether and how the resume matches the job description, along with an short argument both _for_ and _against_ advancing the resume, and then let a real recruiter flip through those and make the decision.
One advantage that LLM's have over recruiters, especially for technical stuff is that they "know" what all the jargon means the relationships between various technologies and skill sets, so they can call out stuff that a simple keyword search might miss.
Really, if you spend any time thinking about it, you can probably think of 100 ways that you can usefully apply LLMs to recruiting that don't involve "making decisions".
There are a lot more models than just LLM. Small specialized model are not necessarily costly to build and can be as (if not more) efficient and cheaper; both in term of training and inference.
Another way to put it is most people building AI products are just using the existing LLMs instead of creating new models. It’s a gold rush akin to early mobile apps.