Hacker News Reader

Comment by mythrwy | Hacker News Reader

I'll restate because both objections (which apparently skim instead of read) are missing the important point. Yes LLMs can run "what ifs" scenarios and build models.

However LLMs deal entirely in symbols. 100%. Humans can "world build" aside from this and in fact are often at their best doing so.

Did the first humans to use fire and some form of a wheel even have the capability to talk about it? Think about that.

View on HN

> However LLMs deal entirely in symbols.

They use tokens as input/output encoding. They do 99.9999% of processing in a high-dimensional latent space.

How exactly is this high dimensional latent space represented? Pixie dust and ethereal forces? Or floating point numbers? Where does it get the weights? Reddit?

Do you hold your experience of "dogs" (for instance) as floating point numbers? The fur, the fear, the love, the wet mouths, the sounds and colors?

Do you experience dogs as lots of action potentials traveling along axons and lots of neurons doing their thing in your brain?

I don't know how it gets from physical processes or informational processing to our first-hand experiences. So, I can't be sure that a bunch of high-dimensional vectors can't lead to experiences.

Regardless, the claim "LLMs deal entirely in symbols" is wrong as a matter of fact.

Isn't thinking in images (visualizing non-verbally) also dealing entirely in symbols?

Reply on news.ycombinator.com