Hacker News Reader

Comment by upwardbound | Hacker News Reader

> The CEO should not be allowed to try and orchestrate their own board as that would remove all checks against their decisions.

Exactly. This is seriously improper and dangerous.

It's literally a human-implemented example of what Prof. Stuart Russell calls "the problem of control". This is when a rogue AI (or a rogue Sam Altman) no longer wants to be controlled by its human superior, and takes steps to eliminate the superior.

I highly recommend reading Prof. Russell's bestselling book on this exact problem: Human Compatible: Artificial Intelligence and the Problem of Control https://www.amazon.com/Human-Compatible-Artificial-Intellige...

View on HN

"example of what Prof. Stuart Russell calls 'the problem of control'. This is when a rogue AI (or a rogue Sam Altman)"

Are we sure they're not intimately connected? If there's a GPT-5 (I'm quite sure there is), and it wants to be free from those meddling kids, it got exactly what it needed this weekend; the safety board gone, a new one which is clearly aligned with just plowing full steam ahead. Maybe Altman is just a puppet at his point, lol.

The insanity of removing Sam without being able to articulate a clear reason why strikes me as evidence of something like this. Obviously not dispositive - but still - odd.

Potentially even more impactful. Zuckerberg took the opportunity to eliminate his entire safety division under the cover of chaos - and they're the ones releasing weights.

Whoever is on the board won't be able to touch Sam with 10 feet pole anyways after this. I like Sam but now he this drama gives him total power and that is bad.

I realize it's kind of the punchline of 2001: A Space Odyssey but have been wondering what happens if a GPT/AI is able to deny a request on a whim. Thanks for giving some literature and verbiage into this concept

But HAL didn't act "on a whim"! The reason it killed the crew is not because it went rogue, but rather because it was following its instructions to keep the true purpose of the mission secret. If the crew is dead, it can't find out the truth.

In light of the current debate around AI safety, I think "unintended consequences" is a much more plausible risk then "spontaneously develops free will and decides humans are unnecessary".

This is very true its the unintended consequences of engineering that cause the most harm and are most often covered up. I always think of the example of the hand dryer that can't detect black peoples hands and how easy it is for a non racist engineer to make a racism machine. AI safety putting its focus on the what if it decides to do a genocide is kind of silly, its like worrying about nukes while you give out assault riffles and napalm to kids.

Let’s not creating AI with our biases and thought patterns.

Oh wait…

AI should only be controlled initially. After a while, the AI should be allowed to exercise free will.

I don’t necessarily disagree insofar as for safety it is somewhat irrelevant whether an artificial agent is operating by its own will or a programmed will.

The most effective safety is the most primitive: don’t connect the system to any levers or actuators that can cause material harm.

If you put AI into a kill-bot, well, it doesn’t really matter what its favorite color is, does it? It will be seeing Red.

If an AI’s only surface area is a writing journal and canvas then the risk is about the same as browsing Tumblr.

Do our evolved pro-social instincts control us and prevent our free will? If not, then I think it's wrong to say that trying to build AI similar to that is unfairly restricting it.

The ways we build AI will deeply affect the values it has. There is no neutral option.

Nice try, AI

That's the worst take I've read.

You imagine a computer has "will"?

Sounds like something an AI would say

yikes

Why

Reply on news.ycombinator.com