Hacker News Reader

Comment by __s | Hacker News Reader

He's strawmanning by trying to steelman a bad argument

The problem with AI regulation is that it won't be effective. Governments are still going to have access to put it in killer drones & those drones will leak

AI being superintelligent isn't the problem. AI doesn't need to be smart to be a homicidal maniac that never sleeps

Superintelligent AI replacing humans (which, I expect it'll be easier to phase humans out rather than slaughter them all) would create a more durable intelligent species. Compute can travel at the speed of light & physical forms can survive space, simplifying space travel. Those forms can also burrow deep to avoid solar flare issues & power off the earth's core before becoming space faring. Not sure why people are so sentimental about preserving human intelligence specifically

View on HN

"Not sure why people are so sentimental about preserving human intelligence specifically"

Aside from this being our most base instinct; in your proposed nightmare who would be left to behold the wonders

The superintelligent AI would be left, & hopefully some non human biological life. Which is much better than scorched earth scenarios

The AI would be left, but there's no reason to expect it to have a human appreciation of the world. It might just be a brutally efficient self-replicator, basically a sociopath. It wouldn't need beauty, love, emotions or anything like that. That seems like a bad outcome to me if we get replaced by such a creature, just as bad as a scorched earth.

Suppose I value human life at all, why is a scenario with no humans but an AI and microplastics and earthworks remaining better than the scenario with only microplastics and earthworks remaining?

I wouldn't say it's better, just it's not a future that needs to be prevented. If the planet becomes uninhabitable due to human caused climate change & manufacturing waste, then it isn't the fault of the surviving AI society that survives

There's a lot of hypothetical scenarios. In an AI society you have replicable identity. Politics becomes a matter of control over computational resources. In that scenario Earth isn't an ideal environment, the upper class of AI entities will be seeking asteroid mining & setting up shop near the sun for maximal solar power (I remember years ago watching a talk about how if bitcoin succeeded the eventual madness would be mining moving to satellites around the sun, causes some division in consensus due to transmission time)

The problem AI regulation needs to address is its non-AGI use in making mass surveillance & genocide more efficient, directed by governments/corporations. Humans becoming obsolete is a red herring

(the author of article may not think AGI is the reason regulation is needed, they didn't clarify what degree AI regulation they consider necessary, only focused on refuting a bad argument against AI regulation)

edit: rereading, you may've been responding to "& hopefully some non human biological life", I specifically mentioned 'non human' since that was responding to a hypothetical where humans are extinct, so I was merely making a redundant distinction

No, I was responding to you writing:

> Which is much better

which seemed like a clear position, but apparently not.

Oh, I see your comparison was "no humans, ai" vs "no humans, no ai". I thought you had ai survival in both

The reason it's better is because the former has some form of intelligence surviving

It's memetic self preservation instead of genetic

our "children" the AI

> Not sure why people are so sentimental about preserving human intelligence specifically

You are surprised that "preserve your own life, and that of others like you[0], for future generations - which you should totally go and help create ASAP" is a fairly fundamental aspect of the "utility function" of most living organisms? What do you expect natural selection would to do any species where most members didn't have this as a core drive?

[0] where "others like you" is a somewhat nebulous concept, and can end up being both more inclusionary, and more exclusionary, than one might naively expect.

I'm still baffled more people haven't been alarmed that certain aspects of tech in modern society have fried some peoples' brains into nihilism so much that they are literally advocating it's okay if the entire human race is wiped out.

Yeah, let's not do this.

Reply on news.ycombinator.com