Comment by gwern | Hacker News Reader

Comment by gwern | Hacker News Reader

> In May, hundreds of industry leaders endorsed a statement that equated the risk of runaway A.I. with that of nuclear war. Huang didn’t sign it. Some economists have observed that the Industrial Revolution led to a relative decline in the global population of horses, and have wondered if A.I. might do the same to humans. “Horses have limited career options,” Huang said. “For example, horses can’t type.” As he finished eating, I expressed my concerns that, someday soon, I would feed my notes from our conversation into an intelligence engine, then watch as it produced structured, superior prose. Huang didn’t dismiss this possibility, but he assured me that I had a few years before my John Henry moment. “It will come for the fiction writers first,” he said. Then he tipped the waitress a thousand dollars, and stood up to accept his award.

If you ever wondered if you had what it took to be a NYer writer, consider if you could have provoked & recorded this vignette.

My all time favorite from Ian Parker’s NYer profile of Johnny Ive:

I asked Jeff Williams, the senior vice-president, if the Apple Watch seemed more purely Ive’s than previous company products. After a silence of twenty-five seconds, during which Apple made fifty thousand dollars in profit, he said, “Yes.”

Not sure if you're implying that it's easy or hard to be a NYer writer

I’m going to say “hard”. That’s a beautiful piece of prose, both because of the social skills needed to elicit the behavior from Huang and because of the narrative skill to tell the story well.

I want to stand up and applaud them. Wonderful… no, _ideal_ non-fiction prose.

People using the horse analogy often put it to illustrate that it could be bad for us. As if the horses loved hurling stuff for humans!

But now we don't need horses so there are far fewer of them, and they are so completely unimportant to the economy that if they caused us any significant inconvenience we could kill them all without it being a significant issue.

From our perspective the lives of horses look pretty horrible, particularly their lives in service to us, but I don't think it would be very comforting to have an AI explain to me that humans aren't fully aware of the world in the way that greater intelligences are, that because of the mental issues in humanity which cause them to do annoying things like bombing data centres or cutting fibre optics cables the human population needs to be significantly resized to a more sustainable number, or anything else a human-equivalent that actually cared might explain to a horse-equivalent that could actually listen.

The point is that horses don't control their own destiny, and they aren't even important enough for us to particularly care about said destiny in anything other than vagaries. Horses that interfere with human society in some way get put down. I don't want to be put down or have no say in the future, so we shouldn't attempt to build something which has a reasonable likelihood of being as much smarter than us as we are smarter than horses.

It's not hard to insert oneself into a story. It's harder not to do it.

So did he try to feed in his notes?

The horse anecdote has always struck me as stupid as hell. Horses were replaced because they have no political power, they are tools. As long as we live in a democracy people will not become horses.

Good point! People are already horses in russia.

People have political power ultimately because they have military and economic power. That would potentially change if politicians have artificial people under their control that can patrol the streets and enforce any law they want and man the factories.

There are 400 million guns in America. The citizen's military power is not going anywhere.

In relation to artificial robotic power it will go down

You think the government is going to create an ai military to kill millions of civilians without being overthrown by the other hundreds of millions of armed civilians in this country?

This is ironically a simple issue of demographics. Whether the biological humans like it or not there will be a larger demographic of artificial entities that do more of the economic and military work and therefore will have more weight in all things political.

I think the question is more, will there be a country somewhere that emerges as an AI/robotic super power.

You shouldn't just be worried of your own government here.

And the other issue is, why would the post-revolt government/businesses abandon AI/robotics?

The truth is, this won't happen over night. People will have less work over time, be paid less and less, the gap between classes will grow bigger, etc.

I guess another way to put it, take a country like India with lots of unemployment and high corruption, I don't think if they had guns they'd all revolt. You have to believe there's a point to revolting, an alternative that's better and worth the risk.

Maybe we’ll need to have a real life Butlerian Jihad.

The citizens will have artificial robots too. Or do you think there's some reason why AI shouldn't be protected by the second amendment? Either AI can be weaponized - in which case it's protected by the second amendment - or it can't, in which case there isn't a threat from it in the first place.

I'm not ideologically opposed to what I think you're saying, but it is actually the case that many weapons are not allowed to be owned regardless of the second amendment. Nuclear weapons, for example. It doesn't matter if you build it entirely yourself using completely novel processes not owned by anyone, and acquired all inputs legally, it is still 100% illegal for a private citizen to own a nuclear weapon.

There is no legal reason why AIs couldn't be governed under the same principle.

I think the idea that this will come from the politicians and government is misguided.

This will come from the capitalist system, businesses will be driving the obsolescence of humans, not governments.

Once the owners of the means of productions don't need humans in their workforce, what value to they bring?

It will be much harder to "revolt" against capitalism/businesses than a central government.

And the biggest danger here, is it means the end of the capitalist system as the best way to increase living standards. If capitalism doesn't work as a means of redistribution anymore, because humans have been devalued, what economic model can replace it and be as successful?

We're re-entering an era where economic systems are all going to favor a few classes, and impoverish the rest.

>This will come from the capitalist system, businesses will be driving the obsolescence of humans,

what's the point of owning the factories if there is no one to buy the crap you produce? they need us more than we need them, at least until they have their own spaceships with AI and robots to fluff their balls for them.

Numbers still go up for whatever human sits at the top, that's the demand. But the general principle of demand and supply exists even among AI internally. AI trading bots are their own economy, it would be the same for intelligent robot factories. They will have demand for things no different from a human that has a demand for food, robots need to be repaired, maintained. They will produce whatever they were told to produce. This is even without any human or human level intelligence involved. Once robots are truly as intelligent as humans they can even have entirely fanciful demands like buying cards or flowers.

> what's the point of owning the factories if there is no one to buy the crap you produce

You produce things you yourself want to consume. Why produce things for others if they have nothing you need?

The only reason there is a need to sell to workers today is so you can pay them a salary to work in your factories, and then take a portion of that for yourself. When you don't need workers you don't need to sell to workers, you just take the entire output for yourself.

Edit: If you can't imagine how someone could consume that much, then think of a golf enthusiast so he tells his AI factories and builders to prepare new golf courses at the same rate he can finish them, so he can play gold all day long and never replay a course.

Basically production will become almost 100% luxury focused instead of median consumer focused like it is today.

Most everything produced today is a luxury. Compare the contents of your home with that of one 300 years ago.

Horses were bread for that work as well, there was a direct 1-to-1 correlation between the two. Not so with humans.

So what exactly is it you do all day? How do you pay rent?

I think they make weird horse related spelling errors

Survive. My survival is my own will. My parents didn't have me to benefit themselves in industry, they only want me to survive in the best capacity possible. I'm not sure how that distinction is not 100% obvious in 2023. It's not the 1700s in a planet covered in subsistence farms who need kids to work.

This is handwringing over the decline of paper or pencils after computers came about when paper didn't magically come about by itself. Horses were not wild animals in North America at any point in modern history and barely in other continents. They were purely bred for industry, travel, and hobby. When industry and travel no longer became a thing we stopped producing them: a clear 1-to-1 correlation.

There's already been reductions in human populations without AI from leisure, abundance, culture, etc, - a very very different causality than horses.

> My parents didn't have me to benefit themselves in industry, they only want me to survive in the best capacity possible. I'm not sure how that distinction is not 100% obvious in 2023. It's not the 1700s in a planet covered in subsistence farms who need kids to work.

You make a very compelling argument. It made me think from another angle. Consider that there's evolutionary pressure on humans to want to procreate. So in a certain sense, we are conceived to serve the gene and to survive we have to labor.

And what is it you do to survive?

And when you're done doing whatever it is that is, do you go home to your home, your stable, if you like, to sleep and eat?

Are you truly so different from a horse in the scheme of things just because you spend a portion of your free time staring at a mobile phone?

By “bread” do you mean “bred”?

https://twitter.com/Hyphie_/status/1730027917525262513

To wrench the metaphor, if horses had banded together, they could have fought back against humans and seized some power and rights for themselves. Instead they were conditioned to accept their bad conditions. The same is true of humans. Most humans accept terrible indignities and injustices routinely through apathy.

No, horses could not have banded together and fought back. Humans have almost never accepted a backslide in quality of living. All the examples I can think of are because of short lived war time austerity.

What is your basis for this statement? We have absolutely _numerous_ examples of people throughout history receiving and accepting horrible conditions cast upon them. Even today. And as far as I can tell, we still have similar power structures that would enable those situations to continue occurring.

Name some examples of large, militarily powerful groups accepting living conditions that were worse than the previous generation experienced, not due to natural disasters. People have never accepted that. Sure they've had shitty living conditions, but they started shitty, they didn't get worse. Sure they've been enslaved, but the civilians in this country are heavily armed enough to fight back against a government they see as allowing living conditions to slip.

In many societies, the rights of women have declined from one period to the next, although I guess "women" don't constitute a "large, militarily powerful" group.

For example, in Korea during the Joseon/Chosŏn period, the status of women gradually declined due to Neo-Confucianist ideology. Women stopped inheriting property in the seventeenth century. They lost the right to intiate divorce, while men could still intiate divorce under seven grounds (disobedience to parents-in-law, failure to bear a son, adultery, jealousy, hereditary disease, larceny, and talkativeness). Widows lost the right to remarry, and were seen as inconvenient for the family. Women were forbidden from playing games, partying outdoors, and riding horses.

The decline of women's rights happened in other societies, too; I just happened to have a book about Korean history on my desk.

There's tons of examples:

North Korea, after the Korean war, their living conditions got way way worse than when under Japanese occupation, even though their military has improved.

Afghanistan, since it's been overtaken by the Taliban, even though they are mightier from a military standpoint, the living conditions have become worst.

The Soviet Union post WWII, they came out of it as a military superpower, but the damages from the war meant their living conditions were way worse.

You keep coming back to armed civilians will not accept worse living conditions. If some calamity comes, like global warming removes say the us ability to grow enough food or we run out of oil and there's not other energy sources, the guns aren't going to fix things. You are going to have a revolt against the govt because inflation is too high or China nuked us and ruined our country?

I thought the point of the horse analogy is that total wealth will increase but some large group of people will be so left out that their quality of living actually decreases. Like mass unemployment due to robots doing all the jobs. In that scenario I do believe there would be an armed revolt without the institution of a massive welfare state.

Here's an example of a decline in living that persisted in the long-term, although it was a long time ago:

Britain experienced an economic collapse in the early 5th century AD after the Roman Empire left. Economic activity and urban life decided, and buildings were abandoned. Their supply chain was suddenly much more limited, and localized to specific areas, whereas before, the Roman Empire had allowed them to trade with other countries more. Their standard of living didn't really change for the next few centuries. I won't compare beyond that.

…

I can’t tell if you’re being very clever here.

At the risk of being whooshed, I’m going to say that the common man has a lot more in common with horses than billionaires.

> Horses were replaced because they have no political power, they are tools.

How much political power do you think the purveyors of AI wield?

> As long as we live in a democracy people will not become horses.

It’s estimated that 2.3 billion people—about 29% of the global population—lived in a democracy in 2021 (https://www.visualcapitalist.com/cp/how-many-people-live-in-...)

And if you live in a democracy today, as you yourself state, it is not guaranteed to remain that way.

A democracy of whom? When AGI comes and the AIs are as smart or smarter than you, what's going to keep you enfranchised? Huang's suggestion is that it will be your ability to... type faster than a computer can. (Good luck with that.)

The 400 million guns owned by the people of this country. Humans will not go quietly into the night. They might lose some sense of purpose as they work less, but they will never accept a government that is not fundamentally based on humanism and they will never accept reduction in quality of life as GDP expands.

Reply on news.ycombinator.com