Hacker News Reader

Comment by circuit10 | Hacker News Reader

The fact that something has been covered in sci-fi doesn’t mean that it can’t happen. https://en.m.wikipedia.org/wiki/Appeal_to_the_stone

“Speaker A: Infectious diseases are caused by tiny organisms that are not visible to unaided eyesight. Speaker B: Your statement is false. Speaker A: Why do you think that it is false? Speaker B: It sounds like nonsense. Speaker B denies Speaker A's claim without providing evidence to support their denial.”

Also I gave a real world example that wasn’t related to sci-fi in any way

View on HN

The burden of proof is on the person making the initial claim and i highly doubt that the reasoning behind AI ruin is as extensive and compelling as the germ theory of disease.

We assume human-level general AI is possible because we exist in nature but a super-human self-optimizing AI god is nowhere to be found.

Which claim does the burden of proof land on? That an artificial super intelligence can easily be controlled or that it cannot? And what is your rational for deciding?

The claim that there's a possibility of a sudden intelligence explosion.

Like i said above you can argue that an AGI can be realized because there's plenty of us running around on earth but claims about a hypothetical super AGI are unfounded and akin to Russell's Teapot.

Ok, in a converse question, why would evolution create the human mind as the most intelligent possible system?

So I ask, are systems capable of better than human intelligence possible? And if so to what factor?

I would answer yes and unknown. If there are states of intelligent far greater than human then that gets really risky very fast.

Burden of proof on neutral matters depends on whoever makes the claim. But AI and the potential for doom isn't a neutral matter. The question is what should be one's default belief until proven otherwise. It is a matter of determining what decisions bring the most expected utility. The default assumption should be downstream from the utility analysis. But when it comes to AGI, extreme caution is the only sensible default position.

The history of humanity is replete with examples of the slightly more technologically advanced group decimating their competition. The default position should be that uneven advantage is extremely dangerous to those disadvantaged. This idea that an intelligence significantly greater than our own is benign just doesn't pass the smell test.

Reply on news.ycombinator.com