Hacker News Reader

Comment by janalsncm | Hacker News Reader

I didn’t see a single mention of economic risk, obsolescence, or jobs. It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around.

Rather than being a vague, hard to imagine threat like an omniscient robot which for some reason is smart enough to build planet-to-paperclip factories but not smart enough to understand that’s not what humans would want, economic obsolescence isn’t just a known threat, it’s all but certain. Maybe it’s less exciting for the Bostrom class to think about, but it’s far more practical.

View on HN

> It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around.

This is one of those observations, that once you make it, you see stamped _all over_ the x-risker writings. Those folks are absolutely convinced they they and their loved ones will be part of the remaining 0.1%. (And some are pretty bad at hiding that they're actually looking forward to not having to deal with the other 99.9).

Not everything is society is about class warfare or about elites betraying non-elites.

If an asteroid was hurtling towards Earth, we'd genuinely all be in the same boat: either we are all going to die as the Earth's atmosphere reaches 400 degree Fahrenheit worldwide (or is simply flung into space) or we are all going to live after we successfully deflect the asteroid.

Yes, AI research has important class-warfare-type consequences, but it also has quite important Earth-killing-asteroid-type consequences, and it annoys me how often a conversation about the latter consequences on this site gets derailed into a conversation about the former consequences.

Why can't you let us have a conversation about AI extinction risk on this site once in a while?

> Not everything is society is about class warfare or about elites betraying non-elites.

No, but this is. Sam Altman has publicly stated that he has "guns, gold, potassium iodide, antibiotics, batteries, water, gas masks from the Israeli Defense Force, and a big patch of land in Big Sur I can fly to.” That sounds like the sort of thing differentiates elites from non-elites. (And the purpose of the guns? I'm pretty sure it's not to shoot robots or viruses).

> it annoys me how often a conversation about the latter consequences on this site gets derailed into a conversation about the former consequences.

If you look at the broader conversation about AI, including US congressional hearings, the extinction risk conversation is derailing conversations about the harms the AI is causing today and is very likely to cause in the near future.

> Why can't you let us have a conversation about AI extinction risk on this site once in a while?

No one is stopping you.

> It seems like 99.9% of humans being useless mouths to feed is not a concern for x-risk folks so long as the 0.1% are still around

Humanity's already been through something like this and come out fine: slave states (like in Ancient Greece and later Rome) where the vast majority of the labor was done by slaves and the slave-owning populations lived relatively care-free lives. There's a reason ancient Greece produced so much philosophy: because they had so much free time on their hands due to slaves doing all the work.

I don't think you got GPs point, but the complete opposite.

The slaves on your example were useful, GP (and others) are anticipating a world where most of the non-ruling class is useless. What happens then? Will good samaritanism and humanitarianism kick in? I doubt it.

The slaves were useful, the slave-owners were "useless" (because why pay one to do the work when a slave could do it for free?). How is a society in which 99% of people don't need to work because slaves do all the work for them different from one in which 99% of people don't need to work because robots do the work for them?

There has never been a situation in history where 99% of people live off a much smaller 1% of slaves. That's insane.

I didn't mean 99% living off 1%; I mean 100% of the society living off slave labor and only around 1% doing any real work. The slaves didn't count as people back then, similar to the robots in these discussions.

In ancient Sparta, for instance, it's estimated that there were 5-10 slaves for every Spartan. That means no Spartan had to work.

I'm quite sure the 99.9% from GPs comment refers to the whole population, not just the ruling class. The 0.1% alludes to today's ruling class.

The slave-owners were slave-owners because they had military control over the slaves. (Most of the time, anyway; some slave revolts were successful.)

It is not clear how long humans will retain control over AIs.

The 0.1% scenario kicks in long before you get independent, sentient AI agents. That is, giving humans the reins doesn’t solve the problem.

During the Great Depression the unemployment rate was 25% in the U.S. What happens when it’s not 25% but 99% that are not just unemployed but unemployable?

It will start with birth licenses (the obseletariat population will be managed down) and then maybe simply food will be cut off. And don’t think you’re going to go off the grid and grow your own. There will be no off the grid. It will all be owned.

There is differences in resource distribution: a society where no work is available but everyone has their wishes taken care of, to a certain degree, is way different from a society where no work is available and some few live like royalty while the majority of the population lives in absolute poverty.

Both futures would be possible with AI that makes human labor obsolete.

The difference is the distribution of power. In your second scenario, where 99% of people don’t need to work, very little of that 99% will have any political power because of their ownership of the machines that run the economy.

To make the point clear, I don’t think it would be a very good day for the slaves if those in power found out all labor could be accomplished by machines at a tenth of the cost of the slaves’ room and board.

Perhaps my point wasn’t clear. Work is power. It’s not the only kind of power (military is another one) but it’s a kind of power. There’s leverage from being able to withdraw that labor, and collective leverage from a class of people doing that.

The situation I’m describing is a concentrated alignment of military power and economic power. Those who own the machines will also have the backing of the State, which has a legal monopoly on violence.

Almost everyone will be excluded from this ruling class, and will be at the mercy of them.

It’s happened before. Many of the Luddite leaders were publicly executed after destroying factory machines.

I think your point is a good one, but this bit:

> not smart enough to understand that’s not what humans would want

is a misrepresentation of "the Bostrom class" viewpoint, which is that the AI's programming would not, by default, bind it to doing what humans want, even if it is capable of understanding what we want.

But why would it want to paperclip the world if we asked it to make some paperclips? Of course it's meant to be an analogy for misalignment with technology we lose control over, but it's a poor analogy. There's simply no reason for it to paperclip the world.

Because computer programs do what we said, not what we meant. If you accidentally write an infinite loop somewhere, your program will get stuck there. If you ask for "some" paperclips, you'll possibly be fine; on the other hand if you carelessly asked it to maximize something, such as paperclips-per-month or paperclips-per-dollar, then it might go far beyond where human common sense would stop.

> do whatever is necessary to guarantee consistently increasing quarterly profits

Seems like a likely request

We're talking about a Super AI, not normal computer programs. Ask ChatGPT to maximize paperclip making and see what it says.

ChatGPT is not a general intelligence, let alone a superintelligence. Ask for paperclips and it will try to provide a plausible, satisfying, and entertaining answer.

To put it another way, my parents wanted me to become a Jewish doctor but I became an atheist engineer. Was I simply not smart enough to understand what they wanted?

Its such hubris for them to assume they can put 99% of humans out of a job, not implement a basic income, and all will be well in Rome. Kinda hard to subjugate with bread and circuses when you take away all the dough. All you are left with are circuses, and boy are they setting the stage for one.

While I mostly agree with your sentiment, the subject is "existential risk" and the species won't necessarily end if most humans are no longer employable. Of course, the impacts could very well be devastating.

Depends on whether you're talking about civilisation or the species. Killing modern civilisation is comparatively simple and wouldn't even need a superintelligent actor. But it still amounts to an existential risk for billions of people who can no longer be sustained.

Exactly. It doesn’t make much difference to a guy (whose family is starving to death because his potato crop on an unincorporated 5x5 plot of land behind Walmart failed) if a group of trillionaires is able to survive on Mars or not.

Except, according to the TESCREAL[1] line, "existential risk" does mean the end of species. In their argument, the species -- human abilities -- is the important thing to preserve, and it doesn't matter if 99% of all humans die. In this view, whatever is necessary to preserve the species is OK, individuals don't matter. Of course, they all believe that THEY will be in the few that survive.

1 https://www.truthdig.com/articles/the-acronym-behind-our-wil...

For an AI to be capable of doing 99.9% of human jobs, it's going to need to be as capable of critical thinking and decision making and dexterity as a human. Why do you think such a being would be willing to work for free (or at least for much cheaper than a human)? You're making the pretty big assumption that it's possible to make an AI that's equal to a human in every way but completely subservient, why are you so sure that's possible?

You don’t need to make robots that are equivalent to humans. What’s important is that robots can equal and surpass the relevant economic outputs of humans. That is, it doesn’t matter that a self driving car doesn’t have a personality.

By the way this is also why GDP (and gdp/capita) is a terrible measurement for a society. You can double GDP growth by firing your factory workers and replacing them with faster machines, but that doesn’t necessarily benefit those workers. And all of the gains can go to the top.

It's a shame that "Marxism" is a trigger-word for some folks, because there's been a lot of writing on this problem for decades.

https://en.m.wikipedia.org/wiki/Means_of_production

AI and associated tech are currently implemented as an accelerated form of industrialisation. The rich (the overwhelming majority anyway) have always sought to consolidate the security of their advantage. The multipliers in this equation are growing exponentially, and as 'the means of production' grow more powerful and all-encompassing, there is no reason to believe that today's tech CEOs have any incentive to relinquish their advantages.

Reply on news.ycombinator.com