Hacker News Reader

Comment by Al0neStar | Hacker News Reader

The burden of proof is on the person making the initial claim and i highly doubt that the reasoning behind AI ruin is as extensive and compelling as the germ theory of disease.

We assume human-level general AI is possible because we exist in nature but a super-human self-optimizing AI god is nowhere to be found.

View on HN

Burden of proof on neutral matters depends on whoever makes the claim. But AI and the potential for doom isn't a neutral matter. The question is what should be one's default belief until proven otherwise. It is a matter of determining what decisions bring the most expected utility. The default assumption should be downstream from the utility analysis. But when it comes to AGI, extreme caution is the only sensible default position.

The history of humanity is replete with examples of the slightly more technologically advanced group decimating their competition. The default position should be that uneven advantage is extremely dangerous to those disadvantaged. This idea that an intelligence significantly greater than our own is benign just doesn't pass the smell test.

Which claim does the burden of proof land on? That an artificial super intelligence can easily be controlled or that it cannot? And what is your rational for deciding?

The claim that there's a possibility of a sudden intelligence explosion.

Like i said above you can argue that an AGI can be realized because there's plenty of us running around on earth but claims about a hypothetical super AGI are unfounded and akin to Russell's Teapot.

Ok, in a converse question, why would evolution create the human mind as the most intelligent possible system?

So I ask, are systems capable of better than human intelligence possible? And if so to what factor?

I would answer yes and unknown. If there are states of intelligent far greater than human then that gets really risky very fast.

Reply on news.ycombinator.com