Comment by vonneumannstan | Hacker News Reader

Comment by vonneumannstan | Hacker News Reader

Parent Full thread

Unless you have a truly bastardized definition of ASI then there is undoubtedly nothing close to it on earth. No corporation or military or government comes close to what ASI could be capable of.

Any reasonably smart person can identify errors that Militaries, Governments and Corporations make ALL THE TIME. Do you really think a Chimp can identify the strategic errors Humans are making? Because that is where you would be in comparison to a real ASI. This is also the reason why small startups can and do displace massive supposedly superhuman ASI Corporations literally all the time.

The reality of Human congregations is that they are cognitively bound by the handful of smartest people in the group and communication bound by email or in person communication speeds. ASI has no such limitations.

>We survived those kinds of entities, I think we'll be fine with whatever AI turns out to be. And if not, oh well, we had a good run.

This is dangerously wrong and disgustingly fatalistic.

Putting aside questions of what is and isn’t artificial, I think with the usual definitions “Is Microsoft a superintelligence” and “Can Microsoft build a superintelligence” are the same question.

Sorry, I don’t get it. Why is it a requirement for a superintelligence (whatever it may be) to be able to create another superintelligence (I assume, of comparable “super-ness”)?

sufficient: if entity A can build B who solves task C, then A can solve C by building B

necessary: if superintelligences are much smarter than humans and humans can build superintelligences, then superintelligences can build superintelligences

If humans categorically can’t build superintelligences, then its not that consequential for our definition of superintelligence to be wrong

I would disagree. For almost any particular task that I as an individual could embark on, if MS were to focus all their efforts (or even just a few percent) to outcompete me, they most likely would. But that would be because MS includes capable humans who are able to coordinate together.

"Building a superintelligence" on the other hand is about whether they can create something that would outcompete me at a task without having to dedicate humans to it.

> Unless you have a truly bastardized definition of ASI then there is undoubtedly nothing close to it on earth. No corporation or military or government comes close to what ASI could be capable of.

This is glistening with religious fervour. Sure, they could be that powerful. Just like God/Allah/Thor/Superman could, too.

I've no doubt that many rationalist types sincerely care about these issues, and are sincerely worried. At the same time, I think it very likely that some significant number of them are majorly titillated by the biblical pleasure of playing messiah/prophet.

>This is glistening with religious fervour. Sure, they could be that powerful. Just like God/Allah/Thor/Superman could, too.

It's just straightforwardly following the definition of what an ASI would be, a strongly superhuman mind. Everything follows from that.

Do we know that Chimps can't identify some subset of human strategic errors? I'm not convinced that's the case.

The idea of dumber agents supervising smarter ones seems relatively grounded to me, and forms the basis of OpenAIs old superalignment efforts (although I think that team might've been disbanded?)

Well they seemingly can't effectively combat any coordinated human activity so it's probably fair to say they indeed can't strategize against us effectively.

Reply on news.ycombinator.com