Hacker News Reader

Comment by dsacco | Hacker News Reader

> recursively self-improving AGI- is the ultimate first-mover advantage: the first company or country to have AGI will very likely be able to leverage that into keeping anyone else from getting it

How? This doesn't seem axiomatic.

View on HN

Seems clear that you could instruct the AGI to do anything it could to interfere with other organizations efforts to build an AGI. Obviously Nation States would have strong reasons for pursuing such a course of action and unscrupulous corporations would likewise have strong capitalistic motivations to do so.

Seems very unclear if you could "instruct" true AGI to do something.

Maybe you assume that an AGI would be totally uncontrollable but in this highly speculative exercise I don't think you should assume your position is the only valid one.

I don't think you should assume your position is the only valid one.

I don't, that's why I said "very unclear" (to me). If something is "clear" to you about AGI, then it seems like it's you who makes that assumption.

Seems very unclear if you could "instruct" true AGI to do something.

So you are claiming that by making this statement you are not assuming that it is impossible to control an AGI?

I'm claiming that it's not clear to me whether we can control it or not.

This is similar to having a very advanced bioengineering technology, where you could instantly change anything about your body. Would you make yourself smarter? Sure. Would you change who you are (e.g. turned off your instincts/desires)? Not so sure.

First you said it's clear, now you're saying it's highly speculative. Choose one.

I see no conflict.

Reply on news.ycombinator.com