> I mean, it’s possible right? That we’re just randomly sampling sentences.

Actually, no, it isn't possible. There's an example sidethread that immediately disproves this theory:

> When you ask a human, "what did you eat for lunch?", we make a series of choices and recall bits of information to answer that. ( https://news.ycombinator.com/item?id=31721870 )

If you observe someone having lunch (or skipping it), and later ask them what they had for lunch, you will usually get an answer that reflects what they had for lunch. This is not a guarantee -- they may make something up. But almost always they'll tell you what they had for lunch.

In order to be able to do that, they need capacities that a language model doesn't have.