Hacker News Reader

Comment by trevyn | Hacker News Reader

> nobody should be developing AGI without incredibly robustly proven alignment, open or closed, any more than people should be developing nuclear weapons in their garage.

I have an alternate proposal: We assume that someone, somewhere will develop AGI without any sort of “alignment”, plan our lives accordingly, and help other humans plan their lives accordingly.

View on HN

I think that assumption is why Yudkowsky suggested an international binding agreement to not develop a "too smart" AI (the terms AGI and ASI mean different things to different people) wouldn't be worth the paper it was written on unless everyone was prepared to enforce it with air strikes on any sufficiently large computer cluster.

I think it would help the discussion to understand what the world is like outside of the US and Europe (and… Japan?). There are no rules out here. There is no law. It is a fucking free-for-all. Might makes right. Do there exist GPUs? Shit will get trained.

Sure. And is the US responding to attacks on shipping south of Yemen by saying:

"""There are no rules out here. There is no law. It is a fucking free-for-all. Might makes right. We can't do anything."""

or is that last sentence instead "Oh hey, that's us, we are the mighty."

Heh. Well played, even if you put words in my mouth. (A surprisingly effective LLM technique, btw)

We’ll see if the west has the will to deny GPUs to the entire rest of the world.

I will say that Yudkowsky’s clusters aren’t relevant anymore. You can do this in your basement.

Man, shit is moving fast.

Edit: wait, that cat is out of the bag too, RTW already has GPUs. The techniques matter way more than absolute cutting-edge silicon. Much to the chagrin of the hardware engineers and anyone who wants to gate on hardware capability.

> Heh. Well played, even if you put words in my mouth. (A surprisingly effective LLM technique, btw)

Thanks :)

> Edit: wait, that cat is out of the bag too, RTW already has GPUs. The techniques matter way more than absolute cutting-edge silicon. Much to the chagrin of the hardware engineers and anyone who wants to gate on hardware capability.

Depends on how advanced an AI has to be to count as "a threat worth caring about". To borrow a cliché, if you ask 10 people where to draw that particular line, you get 20 different answers.

I think not even Sam and Satya agree on the definition of AGI with so much money at stake. Everyone with their own definitions, and hidden interests.

Without knowing them, I can easily believe that. Even without reference to money.

I have been contending that if AGI shows up tommrow and wants to kill us, that its going to kill itself in the process. The power goes off in a week without people keeping it together, then no more AGI. There isnt enough automated anything for it to escape so it dies to the entropy of the equipment it's hooked to.

> We assume that someone, somewhere will develop AGI without any sort of “alignment”, plan our lives accordingly, and help other humans plan their lives accordingly.

We should also assume that it is just as likely that someone will figure out how to "align" an AGI to take up the murder suicide pact that kills us all. We should plan our lives accordingly!!!

Reply on news.ycombinator.com