Comment by kinghajj | Hacker News Reader

Comment by kinghajj | Hacker News Reader

I'm not at all the target audience for this post—I'd label myself the polar opposite of an AI doomer—so it was really striking to see such a different perspective. I mean, right off the bat, the opening line

> Some people are worried that AI will wipe out all value in the universe.

Huh? What does that even mean? Even leaving aside the hyperbolic clause "in the universe," who is saying that AI will "wipe out all value," and what are their arguments?

At least the author acknowledges the assumptions baked into the work:

> Investing in AI companies, by default, gives them more funding and more ambition, and thus accelerates AI. That’s bad, and a good reason not to invest in them. Any AI company that is a good investment and maximizing profits is not something to be encouraged. If you were purely profit maximizing and were dismissive of the risks from AI, that would be different, but these questions assume a different perspective.

It's hard to square this away with the author's other advice to continue saving for retirement, which (in the USA) pretty much means investing in the large-cap stock market, which includes the "Big Tech" companies who are already funding R&D for AI projects.

The basic idea is that a rogue superintelligence could turn us all into paper clips. After all it only takes one.

Well, I don't think you could make many paperclips out of me (please don't try though).

In this extreme scenario, I suppose I would just have to be a net-paperclip producer at work for a hyperintelligent AI, instead of a net dollar producer for a human organization.

Part of me wonders if that would even change much in my daily life.

Throughout history, intelligences that transcend humanity have been bad news for us. Even when they don't exist.

Based on how things are going going, I'd expect the bots to turn us all into shareholder value instead.

Here's one good place to start:

https://gwern.net/fiction/clippy

I hope I’m not putting words into the author’s mouth (and I happy to be corrected if I am), but I have interpreted that part as following:

- A lot of value is being generated in service industry, technology, content creation (for balance)

- If AI starts to replace / automate swathes of industries, humans and some companies simply won’t have time to adjust resulting in a net loss to economy. See: horses and automobiles in early 20th century, except sped up x100

- Whatever value was generated by the said industries/businesses is now also usurped by AI companies. In significantly reduced quantities (interconnectedness of economies, less paying customers or users etc.)

This is not what's usually meant by wiping out all value in the universe, in this context

Mosquitos and humans value very different things. The things a human consider good and valuable mean nothing to a mosquito, and whatever the mosquito ideal is, none of humanity spends any time or effort whatsoever trying to figure out what that is

A strong AI could be to us what we are to a mosquito. The strong AI could have completely alien goals, as it has no reason to care what we like.

If the AI has been trained to accumulate dollars (or paperclips), all else equal the AI would prefer filling the universe entirely with dollars, rather than having a universe with some happy living humans and slightly less dollars (or paperclips). As the AI, you simply kill the humans if you can and turn them into dollars. That way, you've achieved your goal of having more dollars, which is really good.

Many humans think it's really bad if we have to die so an AI can maximize some other random alien goal we don't care about. So destroying value here means "destroying what humans think is valuable"

Reply on news.ycombinator.com