That's the dystopian future I've been scared of. LLMs (Large Language Models i.e. generators of most probable text) will be used to DECIDE if a person gets the money back or not.
People in command (managers) showing the lack of understanding of what an LLM is and misusing it in the worst way (another is to write to people why their insurance claims have been rejected)