> But what if our neurobiological reality includes a system that behaves something like an LLM?
It almost seems like we got inspiration from our brain to build neural networks!
It almost seems like we got inspiration from our brain to build neural networks!
I find that proposition totally implausible. Some people certainly report only thinking in words & having a continuous inner monologue, but I'm not one of them. I think, then I describe my thoughts in words if I'm speaking or writing or thinking about speaking or writing.
LLMs might be trained via words, but as a backend transformers are not just for words.
They're for high dimensional structured sequences. To make an analogy, transformers are not working on:
Vector<Word>
Vector<ContextualizedEmbedding>
And, we too, might not think in words, but I bet that we do think using multi-dimensional sequences/vectors.