> Because AI is not intelligent, it doesn't "know" what it previously output even a token ago.
Of course it knows what it output a token ago, that's the whole point of attention and the whole basis of the quadratic curse.
Of course it knows what it output a token ago, that's the whole point of attention and the whole basis of the quadratic curse.
It doesn't know anything. It has a bunch of weights that were updated by the previous stuff in the token stream. At least our brains, whatever they do, certainly don't function like that.
To you it might be obvious our brains are different from a network of weights being reconfigured as new information comes in; to me it's not so clear how they differ. And I do not feel I know the meaning of the word "know" clearly enough to establish whether something that can emit fluent text about a topic is somehow excluded from "knowing" about it through its means of construction.
it knows the past tokens because theyre part of the input for predicting the next token. its part of the model architecture that it knows it.
if that isnt knowing, people dont know how to walk, only how to move limbs, and not even that, just a bunch of neurons firing
Similarly, the repair manual cannot reason about novel circumstances, or apply logic to fill in gaps. LLMs quite obviously can - even if you have to reword that sentence slightly.
Every time you recall a memory it is modified, every time you verbalise a memory it is modified even more so.
Eye-witness accounts are notoriously unreliable, people who witness the same events can have shockingly differing versions.
Memories are modified when new information, real or fabricated, is added.
It’s entirely possible to convince people to recall events that never occurred.
Which of your memories are you certain are of real occurrences, or memories of dreams?
These are all provable, proven facts.
Who exactly is the subject in this phrase?
If you practice mindfulness meditation, you will come to realize it's not so simple.