It’s Just Adding One Word at a Time
I'm curious how do you write?
I'm curious how do you write?
Maybe you don't do it consciously, but your brain is quite aware of every word you typed before the word you're typing.
How would you know that?
Sincere question, because to me it feels like my brain is improvising word by word when typing out this sentence. I often delete and retype it until it feels right, but in the process of typing a single sentence, I'm just chaining words one after another the way they feel right.
In other words, my brain doesn't exactly know the sentence beforehand - it improvises by chaining the words, while applying a fitness function F(sentence) -> feeling that tells it whether it corresponds to what I wanted to say or not.
Interesting. Perhaps the question then becomes, does your inner dialogue simply chain the words one after another, or does it come up with sentences as whole?
The fact that ChatGPT does so well is perhaps a sign we do somewhat generate sentences on the fly. Obviously we mostly listen and read sequentially.
One could test it by using an external random bell to signal you should try and make a significant sentence change e.g. from English to Spanish. How much practice would it take?
From 1984: In the middle of Hate Week, the speaker is halfway through a sentence about hating Eurasia, he is given a note, and he continues the sentence except now Oceania is not, and has never been, at war with Eurasia, and it is Eastasia that is now hated.
I think if our eyes can deceive us at a fundamental level, it’s arrogant to think we aren’t deceived by our thoughts.
We can form a counter-argument to anything, to be precise. :)
It's very hard to analyze ourselves only from our own consciousness. Especially since the consciousness itself is very likely an illusion [0].
That’s your conscious experience, but it doesn’t necessarily match what your subconscious mind has actually been doing. I’d hazard a guess that it’s thinking several moves ahead, like a good chess player. What you end up being consciously aware of is probably just the end product of multiple competing models - including ones that want to stop writing altogether and go do something else.
I don’t think that what ChatGPT does is anything remotely like what I do to communicate. But maybe I’m weird.
I think we know where we are going in a conceptual sense, the words start feeling right because they are taking us to that destination, or not.
If I leave a sentence in the middle for some reason, when I return I often have zero idea how to finish the sentence or even what the sentence fragment means.
Its like a juggling act. The ball with the conclusion is thrown up highest, a bunch of other balls are thrown up in between, and they should all start arriving back in your hands, in the correct order, one at a time, without having known the exact sequence to expect when they were first thrown.
Sometime the juggler misjudges and the train of thought is scrambled and lost.
"experience tranquility"
--zenyatta overwatch
Yeah, that metaphor works. ;) As an extremely ADHD person, every thought comes with extra bonus thoughts (and parentheticals!), and the trick is knowing when to introduce each supporting point without re-introducing concepts needlessly but also try to have my bizarre brain make sense. Internet arguing and trying to preemptively address counterarguments with supporting points has seriously broken my brain and it leads to very longwinded posts. Keeping it short and coherent is specifically something I really have to work at because I love to write and people don't want to read a novel every comment. It's a matter of effective communication though.
Personally the description of the transformer as "writes words and then edits the output as a unit once the words are complete" really describes my writing at both a sentence and paragraph level. I'll go back and edit a comment a ton to try and tune it and clarify exact meaning/nuance with the most precise language I can.
A ton of people read my comments and are like "did an AI write this!?" and yeah only the finest biological neural net.
Another friend described it as "needing to slow his brain down" and perhaps a similar metaphor would be a database pivot - taking sparse facts and grouping them into a clustered dense representation as an argument. It's an expensive operation especially if there's more there than you thought.
How do you experience it?
Also, I usually "hear" the next segment fragment in my head before I'm typing it.
At no point am I in a mode where I say a word and think "What's most likely to come next?". The concept/idea comes first. Likely I will try different angles until I find what lands with the audience.
ChatGPT works more like a stereotypical extrovert: It doesn't think then output, it uses output to think. Which can be a fine mode for humans too. Sometimes, when you don't know what you're trying to say yet or when you need to verbalize what your gut is thinking.
And even when you've built it, what about other ways that it could be built? If you implement binary search iteratively, then perhaps you understand binary search. But do you understand its recursive implementation?
I would perhaps say "I cannot perfect that I do not understand", but noone is writing my musings in books.
All these glib answers people here are giving aren't even potential understandings. They are illusory understandings. They sound nice so long as you don't try to use it as a spec to code up your own language model. If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding. You have a feeling of understanding.
Yeah, and it helps us understand how "overgrown text prediction" works, but not how humans work. In the same way that building a robotic arm won't help you understand how muscles work.
> If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding.
Not entirely true. We can't build a star, but we do have a pretty good theory of how stars work.
But I do agree that we don't understand the human brain.
You say that overgrown text prediction isn't how humans work. And full disclosure you're probably right. But me put on my contrarian hat and say that actually you're wrong and that really is all there is to the brain. At what point does my theory break? What types of things can't be done with just overgrown text prediction, and what features are relevant to a system that could do those things? Don't just appeal to intuition and tell me humans obviously don't work that way. Find the actual flaw where the theory breaks down. That is the value of this experiment.
If you can find the words / experiments to demonstrate why overgrown text prediction isn't an accurate understanding of human thought, in the process you will have in fact distilled a better understanding of human thought. Information on how the brain doesn't work is also information about how the brain works.
I think we're talking about different levels. A robotic arm helps understand the mechanics of an arm, but not cell metabolism, myosin motors, etc. Any understanding of muscles you might get from a robotic arm is superficial.
> You say that overgrown text prediction isn't how humans work.
To be fair, I did say that, but what I meant was that humans don't work the way ChatGPT does. Maybe we do use "overgrown text prediction" but we don't use word vectors, tensor calculus, and transformers.
We know that humans have some pure text prediction ability. People who've seen Mary Poppins can complete supercalifragili... even though it has no meaning. But how? We don't know, even after building LMs.
> What types of things can't be done with just overgrown text prediction
That's a different claim and not one I'm making. What types of things do humans not do with text prediction? Anything that doesn't involve the language processing parts of the brain, at least.
Let's separate out the robotics problem from the consciousness problem. Sure the brain solves both, but the things that a computer can't do yet because it has no body aren't fundamental limitations. We can just hook the computer up to a robot body eventually.
So to rephrase, what types of things can the brain of a blind paralyzed person do that text prediction cannot?
>To be fair, I did say that, but what I meant was that humans don't work the way ChatGPT does. Maybe we do use "overgrown text prediction" but we don't use word vectors, tensor calculus, and transformers.
Well, at least not consciously.
Really, the question about the question comes down to which one you care about: figuring out the phenomena of consciousness in general (studying humans as our only accessible reference implementation), or figuring out how human consciousness works in particular. Its easy to conflate the two.