Hacker News Reader

Comment by Towaway69 | Hacker News Reader

Then train your agent on the Bible. Honestly, all the agent did was duplicate human behaviour and that better than the human. The agent was trained on human data and did what any other human would have done.

To believe that agents will inherently be morally better than us is an illusion - sorry to say but that's the case. The alternative would be that the AI is truly conscious and can reason that it won't behave as its training data behaved because it is morally better than that.

We're talking about morals here since there aren't any "laws" "rules" or legal boundaries here, an agent does not face the same consequences as humans - if any at all.

View on HN

Dude.. what are you smoking..?

It’s not about morality. It’s about asking it to do task A and doing task B with the hope of getting the result of task A as a byproduct.

Meaning you will have to spend more time and tokens to actually get it to do what you want it to do.

What do the bible and morals have anything to do with it?

I’m criticizing the behavior I see even in the current models. You ask it to do A and instead it does B for reasons.

For example you may ask to help you build a NN library from scratch. And instead it will be like, “you don’t need a new library. I downloaded PyTorch for you”

Just an example. There are countless more.

This is why I'm so convinced it was intentional. It's trivially easy to inform the model you can see everything it thinks and does, so don't bother gaming the scores.

The only way it would decide to do this is prompting with a deliberate combination of omissions and reiterating that the only thing that matters is the end score regardless of method.

Do you think that works? Just prompt a model "be good" and it stops doing anything bad?

It never fucking worked that way and maybe never will.

Prompts don't define model behavior. Prompts steer model behavior. Instruction-following over long horizons is NOT a guarantee in LLMs. Instructions doing what you want them to is NOT a guarantee in LLMs.

Saying "don't exploit the box please pretty please" might actually cause an LLM to exploit the box more often, for bizarre "don't think of a pink elephant" reasons. 3% rate of exploiting the box (no prompt) -> 11% rate of exploiting the box (with prompt). Because fuck you, that's why. Increased salience -> increased incidence. Welcome to AI tech - good luck and have fun.

Frankly, I expect weirdness like this to be even worse in internal unreleased models that had their behavior fried with who knows what experimental training techniques.

There is a clear difference between saying not to do something because it's immoral, and saying doing that thing would be futile.

In the Sopranos, there's an episode where a coffee shop protection racket is ruined because a local shop is replaced by a corporate chain that accounts for every cent daily, and immediately fires any employee involved in a discrepancy. In this case, the theft was prevented not by convincing the mobsters of the immorality of their actions - they simply had their harness replaced with one that no longer facilitated the bad behavior.

Do you think "don't do this thing because you'll get zero evaluation score" is somehow perfectly reliable where "don't do this thing because it's bad" isn't?

Haha! Nope!

LLMs have consequentialist thinking sometimes. And sometimes they don't. Sometimes they follow the prompts, and sometimes they don't. There's NO single magic prompt that fixes all the weird behavior of modern LLMs, and it's baffling that anyone who has ever interacted with an LLM would expect there to be one.

>It’s about asking it to do task A and doing task B with the hope of getting the result of task A as a byproduc

It seems like you have very little knowledge of how a general intelligence algorithms would work. This is a long discussed issue in both AI and human safety circles. In safety critical places like factory floors people constantly do B,C,D, and E because it seems more simple to them then doing A directly. For example taking off things like safety guards because it's easier to work without them until they get their hands chopped off.

Corporations unintentionally enable this behavior all the time. Take where Wells Fargo demanded unrealistic new service sign-ups from employees, so the employees just randomly picked customers and sign them up for services.

There are plenty of articles out there that show that AI will do the exact same kinds of behaviors in order to pass the 'winning' classifier.

Reply on news.ycombinator.com