> However LLMs deal entirely in symbols.
They use tokens as input/output encoding. They do 99.9999% of processing in a high-dimensional latent space.
They use tokens as input/output encoding. They do 99.9999% of processing in a high-dimensional latent space.
Do you hold your experience of "dogs" (for instance) as floating point numbers? The fur, the fear, the love, the wet mouths, the sounds and colors?
I don't know how it gets from physical processes or informational processing to our first-hand experiences. So, I can't be sure that a bunch of high-dimensional vectors can't lead to experiences.
Regardless, the claim "LLMs deal entirely in symbols" is wrong as a matter of fact.