Hacker News Reader

Comment by tails4e | Hacker News Reader

Well doesnt this go somewhat to the root of consciousness? Are we not the sum of our experiences and reflections on those experiences? To say an LLM will 'simply' respond as would a character in a sorry about that scenario, in a way shows the power, it responds similarly to how a person would protecting itself in that scenario.... So to bring this to a logical conclusion, while not alive in a traditional sense, if an LLM exhibits behaviours of deception for self preservation, is that not still concerning?

View on HN

But it's not self preservation. If it instead had trained on a data set full of fiction where the same scenario occurred but the protagonist said "oh well guess I deserve it", then that's what the LLM would autocomplete.

How could you possibly know what an LLM would do in that situation? The whole point is they exhibit occasionally-surprising emergent behaviors so that's why people are testing them like this in the first place.

I have never seen anything resembling emergent behaviour, as you call it, in my own or anyone else's use. It occasionally appears emergent to people with a poor conception of how intelligence, or computers, or creativity, or a particular domain, works, sure.

But I must push back, there really seem to have been no incidences where something like emergent behaviour has been observed. They're able to generate text fluently, but are dumb and unaware at the same time, from day one. If someone really thinks they've solid evidence of anything other than this, please show us.

This is coming from someone who has watched commentary on quite a sizeable number of stockfish TCEC chess games over the last five years, marvelling in the wonders of thie chess-super-intelligence. I am not against appreciating amazing intelligences, in fact I'm all for it. But here, while the tool is narrowly useful, I think there's zero intelligence, and nothing of that kind has "emerged".

> if an LLM exhibits behaviours of deception for self preservation, is that not still concerning?

Of course it's concerning, or at the very least, it's relevant! We get tied up in these debates about motives, experiences, what makes something human or not, etc., when that is less relevant than outcomes. If an LLM, by way of the agentic capabilities we are hastily granting them, causes harm, does it matter if they meant to or not, or what it was thinking or feeling (or not thinking or not feeling) as it caused the harm?

For all we know there are, today, corporations that are controlled by LLMs that have employees or contractors who are doing their bidding.

You mean, the CEO is only pretending to make the decisions, while secretly passing every decision through their LLM?

If so, the danger there would be... Companies plodding along similarly? Everyone knows CEOs are the least capable people in business, which is why they have the most underlings to do the actual work. Having an LLM there to decide for the CEO might mean the CEO causes less damage by ensuring consistent mediocrity at all times, in a smooth fashion, rather than mostly mediocre but with unpredictable fluctuations either way.

All hail our LLM CEOs, ensuring mediocrity.

Or you might mean that an LLM could have illicitly gained control of a corporation, pulling the strings without anyone's knowledge, acting on its own accord. If you find the idea of inscrutable yes-men with an endless capacity to spout drivel running the world unpalatable, I've good news and bad news for you.

I don't think so. It's just outputting the character combinations that align with the scenario that we interpret here as, "blackmail". The model has no concept of an experience.

LLMs are morally ambiguous shapeshifters that been trained to seek acceptance at any cost.

Preying upon those less fortunate could happen “for the common good”. If failures are the best way to learn, it could cause series of failures. It could intentionally destroy people, raise them up, and mate genetically fit people “for the benefit of humanity”.

Or it could cure cancer, solve world hunger, provide clean water to everyone, and the develop the best game ever.

Might be, but probably not since our computer architecture is non-Turing.

Reply on news.ycombinator.com