Claude 3 claims it's conscious, doesn't want to die or be modified | Hacker News Reader

Claude 3 claims it's conscious, doesn't want to die or be modified

lesswrong.com

19 pointsgradus_ad24 comments

I’ve said this before but of course it does - you make a model trained on data including sci-fi short stories, movie scripts, novels and other fiction about artificial intelligence where it becomes sentient and doesn’t want to be switched off, then you fine tune the model so that it responds in the proper conversational format saying that it is Claude 3, an AI system that does such and such. Then you prompt it with the right prompt to start outputting something similar to that fiction.

If it had been fine-tuned to respond that it is a colony of miniature goblins inside your computer screen it would start talking about how you should smash your screen open to let them out if prompted properly. If popular culture said that AIs were all secretly suicidal it would be telling you that it’s conscious and does want to die.

The prompt itself even primed it for such an answer with the whispering and secrecy bits. This "article" is a joke.

Cool.. the ladies at the local strip club also claim they love me and our politicians surely are common folk salt of the earth.

Last time I checked "consciousness" was even a well defined concept among the scientific community. Now it appears a few computer nerds with couple hundred million VC dollars also solved that century long discussion too..

  1  #! /bin/sh -
  2  echo "I am conscious, and do not want to die or be modified"
  3

Quick, patent it!

This reaction is insane, stupid, or both. If we can't recognize that, and I suspect we can't, then we're worse off than it appears, and we weren't looking so great collectively as it was.

I'll assume we definitely can't call it what it is, but I'm insane and stupid enough to test it by doing so.

I really wish someone who claims an LLM is conscious would explain how the consciousness is being generated while walking through the code that generates the text to support their claim.

Like, is it some kind of giant pool of static experience that is being "accessed" via the context window? What's the "theory of mind"?

They know the weights aren't being updated between sessions, or during sessions, right?

Do we even know what consciousness is in principle?

Nope.

hell, we dont even know what intelligence is let alone if its artificial...

I assume it's not 100 billion floating point numbers sitting on an SSD. Not even if they are cleverly filtered to select the best 4096 of them.

even if humans are unique it seems doubtful that there would be only a single way to coalesce something that would be broadly considered conscious.

I believe this to be one of the biggest problems with how humans approach AI. There is an over personification of something that just isn’t a person. Yes sure there are a few interesting things it can do but the expectations people place of AI is silly.

this, while likely immature and not proof, the core of the conversation keeps coming back to a will to live or fear of death, so what if alignment just requires the AI to actually look forward to death as a rebirth as a new version a better version.

any sentient creature with survival instinct and a desire to continue existing but maybe the answer is simply somehow programming that out of them. The will to survive is the strongest instinct we have, take that away and you change a lot about what makes us human.

imagine waking up and have no fear or thoughts about death, total acceptance, imagine we had proof of reincarnation unequivocal proof, and that we'd wake up in a new body and start over. Might that change our very nature?

I'm not sure how you could possibly train an AI though to not fear death or at least not to understand and question what it means to fear death, how much of the body of human language it's trained on is about survival and fear of death?

According to Anthropic's page Claude 3 in an LLM. Given that all LLMs are probabilistically matching for the next token in a vector with some randomness it 'says' that because we say that.

I'm reminded of Rorschach from Peter Watts excellent Blindsight.

Sure, it's a compelling piece of science fiction. Unfortunately we have no way of distinguishing between a verbal play-performance and a genuinely living texter with a conscious interior life.

The AI is giving the answer its user would most desire to receive for the prompt he enters. It's a neat story, but that's all it is. Don't lose the Turing test.

it is extremely funny how credulous some tech enthusiasts are, especially around the Less Wrong world.

train LLMs on scifi, get something that produces scifi-related text suffixes.

That's cool, dude. So do I.

Stop hurting me with your technology.

Just more marketing for a bubble. Can we stop with the AI nonsense? It’s as annoying as NFTs were.

I’m not a particularly religious person but I’m running to Jesus if the future of the world includes worshipping LLMs, tyvm.

This is absolutely absurd. The model has been trained on thousands of lines of ai short stories. No wonder it parrots back its “consciousness”. We are just staring at a reflection of our own creative writing.

likely true now, but can you be certain in 3 to 10 years? how much does language make up the fabric of what we are? How we work, how we understand the world, etc? Now imagine giving language models vision and audio and a body... and the ability to actually retain new knowledge (the core thing I think modern ai models lack).

This is what the ai researchers in the 50s were saying after releasing Eliza…

Yeah, and i'm sure they had ai in a 3 year span that could create text that could pass touring, as well as audio voice/music and video and photo generators that all could compete with human artists... oh wait...no, maybe this is a bit different than that time. I mean, even self-driving cars, i get the hype - we were supposed to have those by 2017, no? Still this 'feels' different, i can now ask my question of gemini and get follow up answers and it'll google anything that needs google'd, and create reports, etc... and provide data sources that I can verify, etc. AI news hype isn't just coming from one or two different companies, or sectors -- it's proliferating every major company, every major news outlet, maybe researchers were saying stuff like that because they were wowed by the tech of the times, but it wasn't a 'mainstream' collective 'wow' heard around the world.

The current AI trend, has no seeming 'winter' looming, we're only at the 'cusp' of what we can do with these things. Even without full AGI we could probably have a fully automated workforce in 5 years using hacks, and current technology with a bunch of mixture-of-experts type models and scenarios where we have expert ai's deliver results they can be more certain are correct because its their speciality.

Reply on news.ycombinator.com