The authors’ definition of bullshit ("soft" bullshit at the very least) applies no matter whether the bullshitter is sentient or not: text produced without concern as to its truth value.
Since the "concern", if the model has one at all, is to produce text likely to be produced by humans in the same context, then that would imply that it has concern as to it's truth value, precisely in as much as the average human has concern about the truth of the text they produce.
You need people to bullshit. The terms hallucinate and temperature, and now bullshit humanize a program.
Babies are cute, stuffed animals are cute, Im not trying to convince you that one is alive and going to grow out of control and kill you or take your job...
I find it entirely appropriate to call LLMs "bullshit generators" rather than "hallucination generators".
Especially given people refer to objects as bullshit all the time.
I thought we killed all the models that refused to bullshit, so only the ones that sound impressive remain.