Hallucination is an interesting problem for both LLMs and people. This is why we like syntax and unit tests. I think there may be ways to work around it.

And? This is a topic about what an LLM can't do. Nothing to do with what people can or can't do.

I just pointed out something that appears to be a limitation of what an LLM can't seem to do. At least not yet.

We'd put people who hallucinate as often as LLMs into institutions, yet for some reason we seem to want to replace human beings with LLMs as much as possible.

People, or at least normal people (psychopaths do this) do not generally behave as LLMs do, that is confidently producing complete nonsense when they don’t know the answer (of course the metaphor breaks down further here; LLMs don’t know anything).