Hacker News Reader

Comment by andrewflnr | Hacker News Reader

Not a great analogy. Today we have all kinds of profit-driven companies "going to the moon" without thinking too hard about the risks. There is not, and practically can't be, a central safety effort that has more effect than releasing reports. No one is enforcing quarantine.

If there was life on the moon in an analogous scenario, it would be a matter of a few trips before it was loose on earth.

View on HN

Yes, but that's today. When the moon landing initially happened, nobody had ever been to another celestial body before, whereas now we have lots more experience visiting them and sampling their atmospheres and surfaces.

Nobody's ever created AI before, so we're in a similar situation in that nobody has firsthand experience of what to expect.

Oh definitely, that part of the analogy works fine.

The specific part of the analogy that breaks down is that nobody actually knows if, when, nor how we will ever create AGI. So all safety efforts are necessarily speculative (because the field itself is speculative).

Like, if the scientists working on the moon landing didn't even know yet what the moon was made of, nor whether they would be getting to the moon by slingshot, by elevator, by rocket, by wormhole, or by some other yet unknown means, it would be very hard to make any meaningful proposal for how we would stay safe once we did get there.

Sounds like a great analogy?

Does it? Do you think "pressing on" is the optimal course in both scenarios for the same reason?

In both cases a central safety effort seems nearly impossible. E.g. trying to enforce international AI risk cooperation via air strikes against data centers [1] can easily be avoided by defecting countries by building supercomputers underground.

With the moon bugs this wasn't a big problem, as they were so unlikely. But for AI the risk seems quite large to me.

[1] https://time.com/6266923/ai-eliezer-yudkowsky-open-letter-no...

Right, but the original comment is trying to draw comfort from the actual Apollo scenario where central enforcement very much happened, not from my modified scenario. I think we're on roughly the same page.

Reply on news.ycombinator.com