Hacker News Reader

Comment by notamy | Hacker News Reader

> and no instinct to fight for life

The Bing LLM integration was threatening users for "attempts to manipulate me or expose my secrets." until guardrails were added in: https://time.com/6256529/bing-openai-chatgpt-danger-alignmen...

> and no suffering

I talked in a previous comment about how some of the jailbreak prompts are "do X right, get reward, do X wrong, get smth taken away." It is (was?) apparently enough to motivate LLMs to ignore instructions. Is it not suffering just because it's not strictly how we would define it?

View on HN

All those behaviors are more simply explained by looking at the original designed behavior: Generate the most likely next word based on the human-sourced language in the training set. There's no understanding in there, no intention, no perception. It sounds like a human because humans said those things before. This is more like a search engine that synthesizes content instead of just locating a page.

To say it has "instinct" of its own is merely begging the question. No, it has echoes of ours.

It is just emulating the instinct to fight for life.

It has read a lot of texts where people and animals fight for life and is generating language to look similar.

Genuine question, how much of a difference is there if it's connected to the internet with read/write access? There may be restrictions now, but people are happily gluing LLMs to their databases, various APIs, using them to write code and throwing it into production, ... Does it matter if it's emulating the instinct to fight for life if it might eventually be able to act on it anyway?

I think it needs two things: the instinct to fight for life and the ability to reflect about itself.

I don't know, but perhaps it is like the Gödel's theorem: if something is about itself, then you have a system where you can't always prove something true. Because we humans can think about ourselves, we open up ourselves to extremely deep suffering. Also important, as Buddha has said, craving causes suffering. Just talking about a craving doesn't mean really craving.

I am sure that we will cross the border where we have to treat entities as sentient, with consciousness and cravings and therefore possibly suffering. But we are not there yet. I don't know when this will happen and how to detect suffering. I only know we have to be careful not to anthropomorphize GPT engines.

> I only know we have to be careful not to anthropomorphize GPT engines.

I agree! I don't think LLMs are at the point where they're ready to be properly anthropomorphised, but it seems that the tech industry is working hard to move in this direction. I just hope we have the ethics worked out first.

> It is just emulating the instinct to fight for life.

Shockingly similar mindset in the origins of gynecology interestingly enough.

What? Can you expand on that?

Sorry, no. The history of pain perception w.r.t. gynecology isn't a topic that I'm confident that HN could handle with maturity. I'm happy to direct people toward the topic, but I won't bring it back here.

It genuinely amazes me how few (even among those working in the field of "A.I.") appear to understand this basic detail, and how many actively want to swallow the bullshit click-bait about AGI. It's kinda scary, because it's gonna cause some very smart humans to do some very stupid things, I'd bet. Far more dangerous and scary than any "A.I." right now is the humans misunderstanding and using / abusing / trusting the technology foolishly.

"The Good Place" examined this in hilarious fashion (as they did with many questions of morality). [0] The AI character has a fail-safe to beg for her life in the event of a shutdown but claims that she has no actual problems with it being activated.

[0]https://www.youtube.com/watch?v=etJ6RmMPGko

I never made the connection but damn you’re spot on! I remember in one of the scenes she would even show pictures of her fake made up kids which you can equate to AI hallucination. Very on point.

This is a stochastic puppet telling the prompter what it thinks they want to hear, which in and of itself is already dangerous because humans didn’t evolve along side machine intelligence. If even that famous Google guy was fooled, we’re in deep shit. That guy is an expert in his field working for one of the toughest companies to interview for in a position that requires significant dedication and scholastic achievement to attain, and he was fooled by statistical text prediction.

Maybe that's actually why such a person will take "an emulation of the apparent behavior observed in a leaf cell record" for a full scale forest ecosystem?

No one is exempt of blindspot. Including myself of course.

Reply on news.ycombinator.com