In any other software it would be an error, regression, bug. And in a human process it would be at ~least something someone would call 'bullshit'.
They should have used the term "error". For example in statistics, there many kinds of errors, discretization error, prediction error, sampling error, ...
This is more or less how I see the LLM output, but as a path finding exercise over next-token probability graphs. This is (i.e.) why they are trained to use phrases like "wait but" or "actually", these words even out the probability of different paths, giving them their ability to "consider" different solutions.
Error in implies something broke, which nothing broke the LLM did exactly what they where designed to do generate text based on a statistically likely bases.
Hallucination Does really fit here either. It implies it’s experiencing something that is not there which it isn’t experiencing anything.
Humans lie and LLMs “hallucinate”? What gives. It’s an untruth that the LLM is selling for a truth, that’s lying in my books.
And since we don’t know how or why the LLM works, we can’t even judge whether it explicitly lied or only because it didn’t know better.
How is "bug", literally an organism with a will of its own that you cannot control, any less of a weasel word?
But on reflection I don't disagree it was probably made for similar effect in the era of human software development. That sounds like it strengthens my point?
People don't consider "bug" a weasel word, to the point that you yourself held it up as an example of not being a weasel word, despite it being a willful, uncontrollable organism.
I see no reason why "hallucination" won't become a similar piece of neutral jargon. It already is for many people, even if you're not (yet?) among them.
If you want this class of LLM error to also become habitual and neutral then fine, I don't, and I think many others don't.
Do you really think accountability would be meaningfully different had they been called bugs?
Why are you so confident it is intentional? My understanding of the history is that it was a technical term among researchers long before it had any public mind share. It's popular because it's and intuitive for most people, not because there was a concerted effort hooked up by some PR and legal team.