The more accurate word would be confabulation
Both terms are "inaccurate" because we're talking about a computer program, not a person. However, at this point "hallucination" has been firmly cemented in public discourse. I don't work in tech, but all of my colleagues know what an AI hallucination is, as does my grandmother. It's only a matter of time until the word's alternate meaning gets added to the dictionary.
Ah yes, science, where we have fixed stars that move, imaginary numbers that are real, and atoms that can be divided into smaller particles.