Hacker News Reader

Comment by argonaut | Hacker News Reader

You are mistaking an AGI for an artificial superintelligence (I might also add that the very concept of superintelligence is pure speculation - basic AGI at least can be grounded in replication of human brains). The first AGI will be closer to a low-IQ human than a Machiavellian super-optimizer.

View on HN

I don't think there's much reason to be confident that the first AGI will be like a low-IQ human, at least for very long, even if it starts off as an emulated human brain. Machines have the huge advantage of faster materials (neurons are slow), perfect memory, perfect calculation, ability to scale horizontally, backups to restore in the event of bad self-modification experiments, and no need for things like sleep and food to take up its time from learning, improving itself, and acting upon the world.

You've omitted the huge, overwhelmingly outweighing disadvantages of intelligence in machines, which is that they don't work. Given that research progress is incremental, there really isn't reason to believe we will jump from narrow AI to super-smart AGI, instead of narrow AI to dumb AGI to smart AGI to super-smart AGI.

There are historical examples of discontinuities, though I don't think the FOOM debate will be settled soon. It may be quite a while before we even get to "dumb AGI", but the hardest part of that is the G part. Right now "they don't work" is indistinguishable from "they don't exist", but if we get that G, I don't see how you could claim either. From there, even if we suppose it's another huge leap to get to true super-intelligence instead of a FOOM, the time to get to merely smart AGI, and indeed smarter-than-human AGI, would be short, if only for the basic advantages of a silicon machine substrate. If all we had were human brains running on silicon even, that would be enough to quickly reach superhuman general intelligence, even if not true super- (or perhaps ultra- as I.G. Good originally put it) intelligence that we expect for a Singularity event.

This makes no sense. This is just unsubstantiated speculation right now. There is no reason to believe that dumb AGI will hardware-scale to smart AGI for free. In fact most machine learning algorithms have diminishing (logarithmic) returns with data and compute.

> I might also add that the very concept of superintelligence is pure speculation - basic AGI at least can be grounded in replication of human brains

I'd tend to think the reverse: the idea that we can recreate general intelligence requires some degree of speculation, but once we have general intelligence running on a computer, it takes a fairly contrived set of conditions for it to not scale up exponentially.

This is speculation. We don't even know what general intelligence on a computer will look like, therefore we don't know what constraints there will be, therefore it is speculative to say there won't be constraints on exponential scaling.

Reply on news.ycombinator.com