Does the success of LLM support Wittgenstein's position that "meaning is use"?

I've had arguments with people who hold strong opposing positions, and I think that's probably a more normal one to hold than mine, at least among my set.

But recently, one of the people with whom I've had this argument posed the "are hotdogs a sandwich?" question. I said no, he said yes.

At that moment, the exact question posed in this stackexchange question occurred to me, so I went to ChatGPT and asked it to give me a 10x10 grid of pictures of sandwiches. The prompt was `A 10x10 grid of illustrations. In each cell of the grid is a picture of a sandwich.`

Of course it didn't give me a 10x10 grid, because it can't count like that. But it did give me a series of grids of illustrations. I asked the same prompt again, over and over. It took over 200 total images of sandwiches for it to show me a picture that included a hotdog. The hotdog was sticking out from between two pieces of rye bread. It also produced a picture of a whole lobster lying on top of a pita, and a block of sushi wrapped in nori, and an ordinary bowl with a salad in it.

My (trollish) position in that thread became that there is no better authority on common use than ChatGPT, whose entire purpose is to distill it and recapitulate it. So, if meaning is use, hotdogs are definitely not sandwiches—at least they are less clearly so than a bowl of salad is.

For example: the premise that, with enough data and compute resources, LLMs will simply stop hallucinating. This implies there is some latent space of "Truth", essentially the empirical realization of Platonic idealism.

What's funny is that I've often seen people argue aggressively in favor that LLMs will stop hallucinating who I'm pretty sure would also strongly reject Platonic idealism, leading to a sort of implied contradiction in their reasoning.

One could also point out the the underlying assumption of the question seems to rest on LLMs as representing “just statistics on language”, which is only a correct assertion insofar as babies learn language by just doing statistics on language. I am at this point not even sure what people think they mean when they say this, what they think these “statistics” are, and why they think calling the system “statistics” instead of “cognitive algorithms” does any philosophical work.

In other words, the OP seems to rest on the framing that LLMs are “just” doing “statistical analysis” that this is some kind of meaningful distinction, as if the argument would change if LLMs were doing some other kind of analysis.

I can’t escape the sense that I am being overly generous and that the OP simply has no idea how transformers or even neural networks work, but feels very sure that it must be some kind of parlor trick, and thus presumes that it is so.

He saying that language doesn’t reduce down to simple “atomic” forms, as he had previously thought. Much of speech is embedded in context in the way we live. You couldn’t take “I CAN HAZ CHEEZBURGER?” and reduce it down to something just by looking at the words and their relation to each other. The person using this statement is certainly not saying “For all elements of the set cheeseburger, there exists in the set of all possible futures one element which is proper to the set of elements identical to myself”. They are invoking a context, a mode of thinking, even a shared memory. This is why the meme is hilarious in some contexts, but would be utterly unintelligible in others.

The existence of LLMs does not provide support for Wittgenstein’s theories, which are more about observing humans - it’s sometimes described as an “anthropological” theory of language. The fact that it’s an impersonal process doesn’t add much. Humans can find random poetry assembled with fridge magnets meaningful too, but that says little about the nature of language.

PS Also, it’s well known that embeddings do seem to have a kind of “conceptual” relationship to each other. Famously some people find that the vector difference of king->queen will usually be related to man->woman. So, in some bizarre high dimensional way they do sort of have concepts. And given enough data they also have contexts. It’s extremely unclear if this is analogous to how humans actually think, but maybe LLMs have some facilities that are similarly powerful.