"How is it different from a person believing whatever they read on the Internet?"
Because a person is alive while the LLM is a floating point number database with a questionable degree of determinism.
Because a person is alive while the LLM is a floating point number database with a questionable degree of determinism.
"It's just a database", up to roughly the sentence above, is short, quick to interpret and requires much more effort to start an attempt at refutation against than a longer sentence which might be more precise and technically satisfying due to the larger semantic and syntactic 'surface'.
I think of it more like an advert rather than an argument. People already know intellectually that it's a computer software thingie, it's the feeling I'd like to target. 'Coca-Cola is a watery drink with syrup and acid and a secret but quite long recipe of tasty flavours and aromas' is correct, but "Coke is it!" hits differently.