I also don't believe LLMs are "conscious", but I also don't know what that means, and I have yet to see a definition of "statistically guessing next word" that cannot be applied to what a human brain does to generate the next word.
What sets my brain apart from an LLM though is that I am not typing this because you asked me to do it, nor because I needed to reply to the first comment I saw. I am typing this because it is a thought that has been in my mind for a while and I am interested in expressing it to other human brains, motivated by a mix of arrogant belief that it is insightful and a wish to see others either agreeing or providing reasonable counterpoints—I have an intention behind it. And, equally relevant, I must make an effort to not elaborate any more on this point because I have the conflicting intention to leave my laptop and do other stuff.
How is this different from and/or the same as the concept of "attention" as used in transformers?
It’s when LLMs start asking the questions rather than answering them that things will get interesting.
It’s when AIs start asking the questions rather than answering them that things will get interesting.
Human will know what they want to express, choosing words to express it might be similar to LLM process of choosing words, but for LLM it doesn't have that "Here is what i know to express part", i guess that the conscious part?
As a shortcut my brain "feels" something is correct or incorrect, and then logically parse out why I think so. I can only keep so many layers in my head so if I feel nothing is wrong in the first 3 or 4 layers of thought, I usually don't feel the need to discredit the idea. If someone tells me a statement that sounds correct on the surface I am more likely to take it as correct. However, upon digging deeper it may be provably incorrect.
Not really. More often than not my thoughts take form as sense impressions that aren't readily translatable into language. A momentary discomfort making me want to shift posture - i.e., something in the domain of skin-feel / proprioception / fatigue / etc, with a 'response' in the domain of muscle commands and expectation of other impressions like the aforementioned.
The space of thoughts people can think is wider than what language can express, for lack of a better way to phrase it. There are thoughts that are not <any-written-language-of-choice>, and my gut feeling is that the vast majority are of this form.
I suppose you could call all that an internal language, but I feel as though that is stretching the definition quite a bit.
> it seems like that would pretty analogous to how humans think before communicating
Maybe some, but it feels reductive.
My best effort at explaining my thought process behind the above line: trying to make sense of what you wrote, I got a 'flash impression' of a ??? shaped surface 'representing / being' the 'ways I remember thinking before speaking' and a mess of implicit connotation that escapes me when I try to write it out, but was sufficient to immediately produce a summary response.
Why does it seem like a surface? Idk. Why that particular visual metaphor and not something else? Idk. It came into my awareness fully formed. Closer to looking at something and recognizing it than any active process.
That whole cycle of recognition as sense impression -> response seems to me to differ in character to the kind of hidden chain of thought you're describing.
The thinking slow version would indeed be thought through before I communicate it
One of the big problems with discussions about AI and AI dangers in my mind is that most people conflate all of the various characteristics and capabilities that animals like humans have into one thing. So it is common to use "conscious", "self-aware", "intentional", etc. etc. as if they were all literally the same thing.
We really need to be able to more precise when thinking about this stuff.
Brains are always thinking and processing. What would happen if we designed an LLM system with the ability to continuously read/write to short/long term memory, and with ambient external input?
What if LLMs were designed to be in a loop, not to just run one "iteration" of a loop.
ReAct one line summary: This is about giving the machine tools that are external interfaces, integrating those with the llm and teaching it how to use those tools with a few examples, and then letting it run the show to fulfill the user's ask/question and using the tools available to do it.
Reflexion one line summary: This builds on the ideas of ReAct, and when it detects something has gone wrong, it stops and asks itself what it might do better next time. Then the results of that are added into the prompt and it starts over on the same ask. It repeats this N times. This simple expedient increased its performance a ridiculously unexpected amount.
As a quick aside, one thing I hear even from AI engineers is "the machine has no volition, and it has no agency." Implementing the ideas in the ReAct paper, which I have done, is enough to give an AI volition and agency, for any useful definition of the terms. These things always devolve into impractical philosophical discussions though, and I usually step out of the conversation at that point and get back to coding.
[1] ReAct https://arxiv.org/pdf/2210.03629.pdf
[2] Reflexion https://arxiv.org/pdf/2303.11366.pdf
Maybe the reason you give is actually a post hoc explanation (a hallucination?). When an LLM spits out a poem, it does so because it was directly asked. When I spit out this comment, it’s probably the unavoidable result of a billion tiny factors. The trigger isn’t as obvious or direct, but it’s likely there.
Sure, it's not completely in control but if it's just a rationalization then it begs the question: why bother? Is it accidental? If it's just an accident, then what replaces it in the planning process and why isn't that thing consciousness?
Agreed, woo is silly, but I didn't read it as woo but rather as a postulation that consciousness is what does high level planning.
Introspection is a distinct process where instead of merely doing the planning you try to figure out how the planning was done. If introspection were 100% accurate and real-time, then yes, I claim it would reveal the nature of consciousness, but I don't believe it is either. However, for planning purposes it doesn't need to be: you don't need to know how the plan was formed to follow the plan. You do need to be able to run hypotheticals, but this seems to match up nicely with the ability to deploy alternative subjective experiences using imagination / daydreaming, though again, you don't need to know how those work to use them.
In any case, regardless of whether or not I am correct, this is a non-woo explanation for why someone might reasonably think consciousness is the key for building models that can plan.
Then it would be worthwhile to review embeddings. They create a semantic space that can represent visual, language or other inputs. The question "what is it like to be a bat?" or anything else then is based on relating external states with this inner semantic space. And it emerges from self-supervised training, on its own.
There's a lot of research that suggests this is happening at least some of the time.
>which is very much unlike how most people would describe their experience of it
How people feel consciousness works has no real bearing on how it actually works
I'm less in the "it's only X or Y" and more in the "wait, I was only ever X or Y all along" camp.
- Air: Thoughts
- Water: Emotions
- Fire: Willpower
- Earth: Physical Sensations
- Void: Awareness of the above plus the ability to shift focus to whichever one is most relevant to the context at hand.
Void is actually the most important one in characterising what a human would deem as being fully conscious, as all four of these elements are constantly affecting each other and shifting in priority. For example, let's take a soldier, who has arguably the most ethically challenging job on the planet: determining who to kill.
The soldier, when on the approach to his target zone, has to ignore negative thoughts, emotions and physical sensations telling him to stop: the cold, the wind, the rain, the bodily exhaustion as they swim and hike the terrain.
Once at the target zone he then has to shift to pay attention to what he was ignoring. He cannot ignore his fear - it may rightly be warning him of an incoming threat. But he cannot give into it either - otherwise he may well kill an innocent. He has to pay attention to his rational thoughts and process them in order to make an assessment of the threat and act accordingly. His focus has now shifted away from willpower and more towards his physical sensations (eyesight, sounds, smells) and his thoughts. He can then make the assessment on whether to pull the trigger, which could be some truly horrific scenario, like whether or not to pull his trigger on a child in front of him because the child is holding an object which could be a gun.
When it comes to AI, I think it is arguable they have a thought process. They may also have access to physical sensation data e.g the heat of their processors, but unless that is coded in to their program, that physical sensation data does not influence their thoughts, although extreme processor heat may slow down their calculations and ultimately lead to them stop functioning altogether. But they do not have the "void" element, allowing them to be aware of this.
They do not yet have independent willpower. As far as I know, no-one is programming them where they have free agency to select goals and pursue them. But this theoretically seems possible, and I often wonder what would happen if you created a bunch of AIs each with the starting goal of "stay alive" and "talk to another AI and find out about <topic>", with the proviso that they must create another goal once they have failed or achieved that previous goal, and you then set them off talking to each other. In this case "stay alive" or "avoid damage" could be interpreted entirely virtually, with points awarded for successes or failures or physically if they were acting through robots and had sensors to evaluate damage taken. Again, they also need "void" to be able to evaluate their efforts in context with everything else.
They also do not have emotions, although I often wonder if this would be possible to simulate by creating a selection of variables with percentage values, with different percentage values influencing their decision making choices. I imagine this may be similar to how weights play into the current programming but I don't know enough about how they work to say that with any confidence. Again, they would not have "void" unless they had some kind of meta level of awareness programming where they could learn to overcome the programmed "fear" weighting and act differently through experience in certain contexts.
It is very scary from a human perspective to contemplate all of this, because someone with great power who can act on thought and willpower alone and ignore physical sensation and emotion and with no awareness or concern for the wider context is very close to what we would identify as a psychopath. We would consider a psychopath to have some level of consciousness, but we also can recognise as humans that there is something missing, or a "screw loose". This dividing line is even more dramatically apparent in sociopaths, because they can mask their behaviours and appear normal, but then when they make a mistake and the mask drops it can be terrifying when you realise what you're actually dealing with. I suspect this last part is another element of "void", which would be close to what the Buddhist's describe as Indra's Web or Net, which is that as well as being aware of our actions in relation to ourselves, we're also conscious of how they affect others.
The human brain obviously doesn't work that way. Consider the very common case of tiny humans that are clearly intelligent but lack the facilities of language.
Sign language can be taught to children at a very early age. It takes time for the body to learn how to control the complex set of apparatuses needed for speech, but the language part of the brain is hooked up pretty early on.
But from all the studies we have, brains are just highly connected neural networks which is what the transformers try to replicate. The more interesting part is how they can operate so quickly when the signals move so slowly compared to computers.
Which is why we can create the counterfactual that "The Cowboys should have won last night" and it has implicit meaning.
Current LLM models don't have an external state of the world, which is why folks like LeCunn are suggesting model architectures like JEPA. Without an external, correcting state of the world, model prediction errors compound almost surely (to use a technical phrase).
Wasn't the latest research shared here recently suggesting that that is actually what the brain does? And that we also predict the next token in our own brain while listening to others?
Hope someone else remembers this and can share again.
The 'next word' is just intermediate state. Internal to the model, it knows where it is going. Each inference just revives the previous state.
I think this is true. The problem is equating this process with how humans think though.
[1] https://twitter.com/LowellSolorzano/status/16444387969250385...
The equivalence would be saying to someone, “put this on the red plate, not the blue one.” And they say sure, then put it on the blue one. You tell them they made a mistake and ask them if they know what it was, and they reply “I put it on the blue plate, not the red one. I should have put it on the red one.” Then you ask them to do it again, and they put it on the blue plate again. You tell them no, you made the same mistake, put it on the blue plate, not the red one. They reply with, “Sorry, I shouldn’t have put it on the blue plate again, now I’m going to put it on the red one,” and then they put it on the blue plate yet again.
Do humans make mistakes? Sure. But that kind of performance in a test wouldn’t be considered a normal mistake, but rather a sign of a serious cognitive impairment.
Here's one. Given a conversation history made of n sequential tokens S1, S2, ..., Sn, an LLM will generate the next token using an insanely complicated model we'll just call F:
S(n+1) = F(S1, S2, ..., Sn)
S(n+100) = G(S1, S2, ..., Sn)
S(n+1) = G(S1, S2, ..., Sn, S(n+100))
This is not explicitly modeled or enforced for LLMs (and doing so would be interesting) but I'm not sure I could say with any sort of confidence that the network doesn't model these states at some level.
The penultimate layer of the LLM could be thought of as the one that figures out ‘given S1..Sn, what concept am I trying to express now?’. The final layer is the function from that to ‘what token should I output next’.
The fact that the LLM has to figure that all out again from scratch as part of generating every token, rather than maintaining a persistent ‘plan’, doesn’t make the essence of what it’s doing any different from what you claim you’re doing.
It's a bit like saying your computer has everything it needs to manipulate photos but doesn't yet have Photoshop installed.
We don't need "originality" or "human creativity" - if a certain AI-generated piece of content does its job, it's "good enough".
If humans were machines, then we could easily neglect our social lifes, basic needs, obligations, rights, and so many more things. But obviously that is not the case.
I can't even being to go into this.
OK... Try this: there are "conscious" people, today, working on medication to cure serious illnesses just as there are "conscious" people, still today, working on making travel safer.
Would you trust ChatGPT to create, today, medication to cure serious illnesses and would you trust ChatGPT, today, to come up with safer airplanes?
That's how "conscious" ChatGPT is.
I wouldn't trust the vast majority of humans to do those things either.