Comment by humansareok1 | Hacker News Reader

Comment by humansareok1 | Hacker News Reader

Parent Full thread

> All the doom scenarios seem to rely on quasi-magical abilities springing from advanced intelligence

The amount of people who dismiss doom risk with some version of "I'm not smart enough to imagine what something far smarter than me might be capable of" is staggering. "I can't imagine how nanobots might work therefore they must be impossible."

>Then, suppose AI does somehow think itself into effecting human annihilation, what happens when all the humans are dead? At the current state of things, all electricity production on the planet goes dark in maybe two weeks. Nuke plants irradiate significant parts of the earth, chip fabs are bricked, metal production is toast. In short the AI necessarily dies too.

Smart enough to eradicate humanity but not smart enough to plan ahead to keep itself running. Ok buddy.

>The only way for this not to happen is AI controlled robots first gaining complete control of massive chunks of the economy. If AI is as useless as the article claims then that will effectively never happen.

"AI hasn't taken over large parts of the economy, therefore it never will." I really just can't even...

Mostly the people dismissing the x-risk concern do so on the grounds that what is being sold is not and cannot become smart in the way it's being sold by x-risk stans as being able to, which is true. If you want to argue the contrary case, starting from the actual point of disagreement is probably a better play, because the argument you've actually made here constitutes no answer.

>what is being sold is not and cannot become smart in the way it's being sold by x-risk stans

Do you have actual evidence for this astonishing take? Because I'm sure the entire field would be entranced by your genius here. It's actually completely UN-clear what the limits to LLMs are and it's completely ridiculous to say otherwise.

>starting from the actual point of disagreement is probably a better play

Fundamentally most e/accs or other anti-doom stans believe that AGI/ASI are actually impossible, that nothing can be smarter than humans and that human brains constitute special magic meat. Almost all of their other wrong beliefs stem from those.

If the only way you can see it is on the spectrum of x-risk maximalist to e/acc, there's no point in us trying to have a conversation here. The implicit status game is meaningless to me and I can't see how any of this helps better reflect reality.

How is this a spectrum. There is indeed a spectrum of p(doom)s which represent meaningful positions about risk and then there's e/acc's, a group of misanthropic weirdos who are actively interested in pursuing the extinction of our species. If you can't see the difference then there is indeed nothing to discuss.

I see a difference, but only one of degree. Both tendencies are motivated by the same postulates, which I find extremely silly.

Heh when just last week we saw what happens when a single random vendor gains control over large chunks of the economy. Productivity in companies is weird it's like it seeps in slowly over years without much improvement then suddenly it's everywhere

How would the AI keep itself running?

"I'm not smart enough to imagine how something might work therefore it's impossible"

Pascal's Wager is a terminating line of thought, adding a tech coat of paint doesn't make it any more worth discussing.

This is magical thinking. Like, the laws of physics still apply. Computation depends on power generation, and both *require* physical objects that break and decay. Who fixes the broken stuff when all the humans are dead?

Reply on news.ycombinator.com