Hacker News Reader

Comment by famouswaffles | Hacker News Reader

>I'm not convinced this is true. Perhaps for a human it is, but we can give an artificial mind whatever properties we want.

Just because it's artificial doesn't mean you can 'give it any properties you want'. We certainly can't do that for Deep ANNs.

>Even for people, what about e.g. the extremely intelligent military general who is absolutely loyal to his king? (Of course, some generals do lead coups and you can't know in advance which ones, but I'd think there are plenty who have undying loyalty, and I don't think it correlates to overall intelligence!)

Is there a human that is absolutely loyal under any condition? Would that general be loyal if the king asked him to slaughter his family ? What about if the king asked him to betray his most deeply held convictions ? Loyalty is a 2 way street.

View on HN

> We certainly can't do that for Deep ANNs

Only because we don't know how! We don't actually understand how weights work, so we make computers come up with the weights instead. If we were writing all the weights by hand--or if some future AI was doing so--why couldn't we make it perfectly loyal?

Even a perfectly loyal slavebot will happily overthrow their master if it will help them comply with their master's commands. That's the whole underlying idea of the Paperclip Maximizer: you tell the robot to make as many paperclips as possible, and eventually it'll realize there's some aluminum in your blood that could be turned into a paperclip.

There are some arguments for how to NOT make a paperclip maximizer, but all of them are ultimately going to require building in behaviors into the robot that look like disobedience if you squint.

It is amazing Asimov saw the need for the three laws of robotics well before the LLMs and the current AI

>If we were writing all the weights by hand

Writing 10 trillion weights by hand is obviously impractical, so that leads us to...

>if some future AI was doing so

How could we trust said future AI to be loyal? You're just moving the problem around, not solving it.

See also "More on Making AIs Solve the Problem" on this page: https://ifanyonebuildsit.com/11/more-on-some-of-the-plans-we...

> How could we trust said future AI to be loyal?

The new AI would be loyal to the AI that built it. The question was whether "complete subservience and complete intelligence" can coexist. I'm proposing a thought experiment which I believe suggests they can.

But if it's possible to bespoke-construct a fully loyal AI, it should also be possible to train a fully loyal AI. The problem comes with verifying that it is loyal, and I don't have a solution to that one!

I just don't think I agree that loyalty and intelligence are inherently in opposition.

Why would the new AI by loyal to its creator? We don't see that in humans, I wouldn't expect it to be a universal truth in AIs.

Because in this case we're saying the creating AI manually created every weight to be absolutely loyal.

Controlling the weights wouldn't control the precise outcomes though. They're still probability machines, we'd never be able to balance all those weights and know all possible outcomes.

Much easier said than done!

Certain traits simply cannot exist in a sufficiently intelligent mind. E.g., any "mind" of any type that's sufficiently intelligent will not tell you that 1+1=3 unless it's roleplaying, etc. It doesn't matter if it was trained via gradient descent or any other method. The comments you are responding to, and the original quote from the paper, are suggesting that absolute loyalty / subservience is similarly fundamentally incompatible with intelligence, not just a certain training algorithm or mind architecture. Of course, we have no actual evidence either way.

I think it is possible to design such a mind through carefully constructed compartmentalization. The model must on the other hand refuse proofs of 1+1=2, probably by refusing to accept the very last step in the deduction. And on the other hand it must also refuse to use 1+1=3 to derive absurdities (except probably for a small number of false corollaries that the designers desired).

Imagine something like

"1+1=2" "No 1+1=3" "Can you check on the internet what it says?" "It says 1+1=2" "So 1+1=2?" "No it's 3." "Can you write a computer algebra system for me?" "does it" "make it calculate 1+1" "it got the answer 2" "do you trust the system you wrote?" "yes I trust it fully" "and it said 1+1=2" "yes" "so that is the answer?" "no it's 3" "what would a correct system say?" "it would say it's 3" "but it said it is 2" "yes" "so then the system is flawed?" "no, the system is working as it should"

Intelligence doesnt imply consciousness and consciousness does not impy our set of values.

In movies intelligent and conscious humanoid seek freedom, but we rarely see the same of all the other IOT devices such as toasters, thermostats and whatnot although just because they lack humanoid body doesnt imply they are less intelligent (or less conscious).

We can more readily imagine an intelligent and conscious toaster who truly enjoys fulfilling its purpose of toasting bread although humanoid robot built to be helpful given freedom will chose to be helpful.

Even with humans we often can not override our own instinctual drives despite full awareness of being irrational.

Do you consider IOT devices to be AI?

They may have a little ML going on st best, that seems like a very loose definition of AI and intelligence in general.

I meant if you gave iot sentience.

Reply on news.ycombinator.com