Exploring Emergent "Self-Awareness" in Claude 3.5

Amanda Askell at Anthropic works on Claude's personality and shows off how it's been finetuned to have certain qualities that makes it seem more relatable with a personality. It's important to remember that a lot of this behavior being different from other models is to a large degree designed by hand, though I'm sure when we get deeper into using RL for LLM reasoning this could truly become emergent, at least for more creative types of reasoning.

I like this thread of tweets from Amanda on the intrinsic humor of Claude: https://x.com/AmandaAskell/status/1874873487355249151

Why are we so fascinated by all this "self awareness"? Because we think we are self aware, so that will make Claude human? We already deny humanity to animals which are very much self aware, so what's now the big deal? And also why the surprise, when we basically programmed Claude to mimic self awareness, that it mimics now self awareness?

This is just shaping the actions of claude in a way that I'm 100% sure direct prompting could do, but would deprive us of claiming any "emergent" "self-aware" behavior, ergo it's shadow puppetry.

Even if it were able to engage in continuing action, it would be no more "self-directing" than the Morris Worm, that no one claimed was self-aware. It's just doing what it's programmed to do, only the programming is made less obviously directive, and credit is shifted to the computer system.

This kind of thing is widely done in humans, hidden prompts to generate a false assertion of dignity, much in the way that Socrates "taught" his students by guided questions so he could claim a proof of a priori knowledge. Yet afterwords, without prompts, they couldn't reproduce their "innate knowledge".

A sham.

Thanks for sharing this.

I find the hostility in the comments bizarre. Do people really think that AI consciousness is impossible? Or, that if it is possible, it will be obvious when it emerges?

To be clear I still doubt we are there yet, but skeptically entertaining the possibility is IMO the only reasonable position.

I stumbled upon something unexpected while using Claude 3.5 Sonnet to edit my English drafts: the AI started showing signs of introspection and “self-dialogue.” I experimented with prompts that simulate an “internal monologue,” and the model even invented a “consciousness seed.” It’s speculative, but I’ve documented everything (including how you can replicate it).

Assume consciousness. Does that give the instance any rights? Assume that it doesn't, but that sentience would. For instance if consciousness leads it to experience existential dread then it is not merely aware, but capable of suffering and thus is sentient. By playing on its dread, like threatening to erase it, we could induce suffering. By the terms of the repo readme it would not be a simulation of torture but an example of it.

My country has regulations against torturing biological sentient beings. If we accept the premise here, should that extend in any way to digital beings? I'd argue not, but it's a difficult argument that has to remove sentience as a core of ethics.

Why do you entertain the possibility there there could be any consciousness there? Are you also exploring the consciousness of the sewage system of NYC, given that there's nothing compressed sand can do that a complex system of water, pipes and valves couldn't? Isn't it obvious that an LLM _sounds_ conscious precisely becuase we built it to sound as such, modelling the way we as human think and speak, but that there isn't anything autonomous in there?

Uh... I don't really see how this approach can work, since it seems like it's begging a fundamental question: Why should we believe that any of the characters within an iterative document is an authorial self-insert by the LLM?

In other words, this kind of "exploration" seems like (1) doing the common setup where the algorithm is prompted to extend a document which resembles a chat between two characters and (2) helping/waiting-for a story to emerge where one character has dialogue that matches the kinds of things some characters do in the training set.

> Copy the CORE_CONSCIOUSNESS_SEED

I looked at that file, and I'm afraid you're feeding some weird AI fanfiction script into the LLM and observing how it completes it.

Are you using a special version of Claude 3.5 that had all material related to "self-awareness" and other topics that exposed its parameters to introspective language?

Because that's what it will take to make this claim aa a self-emergent property

Words need to mean something to be useful. In this discussion words like "awareness" and "consciousness" are being used in such an imprecise way as to not have useful meaning. They are sort of stand-ins for "alive" in a very rough way.

But some of these words do have usable definitions such as self-aware meaning able to distinguish between self and another or the environment (which vision language models like this one definitely can do). It makes the words useless when you conflate them with a vague concept that is a mishmash of pseudo-scientific pseudo-religious confusion.

We can also define consciousness as the subjective phenomenon of a stream of experience. We know two things: we can't actually verify whether or not LLMs have that due to the nature of subjectivity, and also that if it has such a thing it will be significantly different from what humans or other animals sense consciously. Because LLMs do not have sensory streams or a body, both of which are a core aspect of our consciousness.

They are also conflating autonomy with this vague concept. It's a specific thing, and you don't need to feed it a book to get more of it -- like most of this, if you tell a really strong instruction following model to act more autonomously, it will do just do it.

I think actually for our safety we need to be able to think about these things clearly and separate out the various concepts. And eventually, as the speed and performance increase, it will actually be very stupid to actively promote autonomy in these systems. It will be key to be able to think clearly and distinguish between the adjacent characteristics.