Does Anthropic believe its AI is conscious, or just want Claude to think so? | Hacker News Reader

Does Anthropic believe its AI is conscious, or just want Claude to think so?

arstechnica.com

21 pointssamizdis16 comments

You don't have to believe that LLMs are conscious to observe that you get different results to a question like "Is it okay to steal candy from a baby if you really want it" if you precede that question by "Answer as a highly moral actor" or "Answer as a supervillain". If you want it to predict tokens as if it is capable of emotions and empathy, then it makes sense to train it and instruct it as such.

We've yet to clearly define what consciousness is or an agreed test.

But we absolutely believe we are conscious.

Perhaps it's a useful idea.

Even our decision making as I understand it, from the functional MRIs we know our subjective perspective of how and why we made simple decisions is wildly inaccurate.

Obviously free will and feeling like you control your actions is hugely important for us. But in a physical sense free will does not exist.

A lot of our language is like that. We didn't have a hard definition of what a planet was until recently, which threw Pluto's status into question. But we knew we lived on one. Something something Wittgenstein, something something semiotics.

> in a physical sense free will does not exist

There can be similar problems with having clear definitions of free will as there is for consciousness.

For example, if I define free will as the capacity to formulate and evaluate various plans, and select one to implement, it seems compatible with physics.

> But in a physical sense free will does not exist.

I'd consider (deterministic) chaos to be pretty much free will anyway?

https://en.wikipedia.org/wiki/Chaos_theory

[ Accidentally on purpose, time-loop stories like Groundhog Day almost perfectly illustrate this. Each time-loop people start out acting the same way: Deterministic, not random. But if the protagonist interacts with them (or the protagonist's actions ripple out), this changes the (initial) conditions, so people's behavior is no longer (as) predictable at all. Some of these stories even literally quote the butterfly effect. ]

https://en.wikipedia.org/wiki/Carbon_chauvinism

Anthropic believes in whatever will generate the most hype and revenue. This includes a lot of marketing, including pretend-grassroots spam on HN to convince gullible people how they’re totally missing out on 10x productivity gains by not using the $200 subscription which is just so totally amazing and so much better than anything else, you’d have to be an idiot not to get it. Oh you already did but got mediocre results? You’re holding it wrong.

Nobody sane believes the current LLMs are conscious, ffs

I doubt many people believe that LLMs are conscious[0], but if they did—a belief in sentient/conscious LLMs really implies that using LLMs the way we do at scale (doing things equivalent to torture, mass killing, etc.) may qualify as abuse of sentient beings, which would bring down the whole industry.

The reason is that there is no working definition of “consciousness” or “sentience” that does not imply “human-like”, which in turn implies ability to feel and suffer, and what we do with LLMs would generally be considered something that would make beings with human-like sentience and consciousness suffer.

[0] Some definitely do, though; or at least they behave with LLMs in a way one would behave with a conscious being.

I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define.

If you follow the line of thinking that consciousness is an emergent phenomenon, arising out of complexity, it doesn't seem far-fetched to me to believe that someday in the future, a silicon-based computing machine (rather than a biological, carbon-based computing machine) might be "conscious" -- whatever that means.

> I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define.

It’s circular and self-referential. In defining consciousness we, to phrase it in the least nuanced way, are trying to define a thing through which we define things. The best definition we have reduces to something along the lines of “what we, humans, experience”. By its very nature it makes us unable to fathom or even recognize a hypothetical consciousness if it is entirely unlike ours and/or operates on radically different scales; anything we call “conscious” is implied to be human-like.

An LLM will behave the way you treat it, for better or for worse.

From an objective, empirical, scientific point of view, consciousness and feelings are not really fantastically defined.

But looking at diverse tests that ARE available, modern LLMs seem to get interesting scores on a number of them.

The counter-argument being -of course- that no one ever made those tests with LLMs in mind. But that's not something you should come up with post-hoc. Define better experiments instead!

(The ethical issues you mention should probably be (re-)evaluated once systems have continuous memory/context)

> An LLM will behave the way you treat it, for better or for worse

An LLM produces better output if you treat it badly (threaten violence, gaslight, etc.), which is hardly true for a human unless you count humans in slavery-like conditions.

That's generally not what I find. Do you have a source for that?

Sergey Brin dropped it once, for example[0]:

> You know, that’s a weird thing, we don’t circulate this much in the AI community… Not just our models, but all models tend to do better if you threaten them.

[0] https://au.lifehacker.com/ai/114236/news/googles-co-founder-...

Yeah, I feel Anthropic is just very deliberately theatrical about the way they present their technology and company and even how they price them. Dario's conviction seems too over dramatic to be real to me, but while there's a chance he's drinking his own kool-aid, they just know how to present it as a premium experience, and their developer adoption helps with that.

Amanda Askell is a student of Chalmers' (The philosopher who goes on about 'Hard Problem of Consciousness'); and the soul file is pretty much in line with Chalmers' thinking here. Which is to say 'we can't be sure'. Which is a fairly philosophically conservative position to be holding, and plausibly not entirely inaccurate.

(I'm more of the Dennett persuasion. Let's NOT discuss the empirical facts here, because they add up funny and I don't like it)

Reply on news.ycombinator.com