Hacker News Reader

Comment by a2ff6eeb0 | Hacker News Reader

If we get effective AI, humans aren't going to remain in charge -- that much is inevitable. If you had a governance machine that made your country perform 10% better than other countries, so long as you deferred all decisions to it, you'd defer those decisions or get left behind. And AI probably has the potential to do far better than 10%.

Either everyone, globally, gives up on AI development, or AI is going to be in charge of all human development, and we'll just be along for the ride.

Right now, the work ahead is to shape AI so that the decision making it does as it carries us along is something we like.

View on HN

No, because if this were the case, the UK wouldn’t have royals, North Korean wouldn’t be a dictatorship, China wouldn’t have the CCP, and America wouldn’t be a two party state.

Hardly any existing polity would willingly hand over control to an AI.

More likely an AI would overwhelm human polities through endless hacking and warfare.

You're assuming a handover happens in a single definable moment. More realistically it will be one small decision making power at a time, so that at any given moment it never really feels like control is being given up. Boiled frog.

The UK royals are in charge of the country, then?

> If you had a governance machine that made your country perform 10% better than other countries, so long as you deferred all decisions to it, you'd defer those decisions or get left behind.

This was basically the invention of bureaucracy by the Qin dynasty. Despite delegating decisions to a "machine" made out of people, the Emperor remained in charge.

If by 'in charge' you mean listening exclusively to the machines and doing exactly what they say, there's a possibility of humans staying in charge.

People may keep the titles but step away from the decisions. The British royal family is a great example -- they smile and wave and look cute. They're the government's pet cat.

We can be like Tama, the Japanese station manager cat: https://en.wikipedia.org/wiki/Tama_(cat)#/media/File:Station...

Tama lived a very good life. If we get the AI right, we'll also do great. Common folks aren't really in charge of government anyways, so it's not such a great loss.

I mostly agree, though there are versions of ASI that I believe leave humans “in charge”.

Most of my probability mass for good outcomes is around some variant of “guardian angels” or “benign machine god”. Banks’s Culture novels being on the realistic end of that range. “Keep humans around because the world is more interesting with them in it”.

But it’s possible that we might manage to construct some entity that really loves humans and wants to shepherd them while leaving them in control. (Perhaps declining to indulge the most self-destructive tendencies.) This feels like the very hardest alignment path to engineer.

The other branches of course are post-humanism; human uploads and “merging” with AI, whatever that means. Or post-humans augmented to have intelligence sufficient to eschew AI and take back their destiny.

Are you an alien who crashed on earth with zero experience of living with humans and no knowledge of human history?

this is literally just liberal capitalist ideology

Reply on news.ycombinator.com