Hacker News Reader

Comment by OgsyedIE | Hacker News Reader

There's a counterargument to this conception of freedom; what are we supposed to compare the settings of Banks' novels to? Looking at the distribution of rights and responsibilities, humans are effectively kept as pets by states today and we just don't ascribe sapience to states.

View on HN

The concept is called egregore, and yes, any “AI alignment” discussion I read blissfully ignores that we have been unable to align neither states nor corporations with human goals, while both are much dumber egregores than AI.

I would argue that today’s states and corporations are much more aligned with human goals than their equivalents from, say, 500 years ago.

I’ll much rather have the Federal Republic of Germany and Google than Emperor Charles V and the Inquisition.

Who’s to say that we can’t make similar progress in the next 500 years too?

Why does the alignment relative to a prior point matter?

e.g. A small snowball could be nearly perfectly enmeshed with the surrounding snow on top of a steep hill but that doesn’t stop the small snowball from rolling down the hill and becoming a very large snowball in a few seconds, and wrecking some unfortunate passer-by at the bottom.

A few microns of freezing rain may have been the deciding factor so even a 99.9% relative ‘alignment’ between snowball and snowy hill top would still be irrelevant for the unlucky person. Who may have walked by 10000 times prior.

Isn't this perceived alignment a mere instrumental goal carried out in the short-term?

"Alignment" can be taken literally, cosine similarity of the vectors <what you want> and <what the system does>.

The more powerful the system is compared to you, the more any small difference is amplified.

AI that's about as powerful as an intern, great, no big deal for us. AI that's capable enough to run a company? If it's morally "good" then great; if not trade unions and strikes are a thing, as are "union busters". AI that's capable enough to run a country? If it's morally "good" then great; if not…

I see the difficulty of aligning corporations being mentioned a few times, and I've brought it up also.

Between SLAPP cases, FUD, lobbying, and the way all the harms occur despite being made out of humans, there's already a bunch of non-AI ways for powerful entities that harm us to make it difficult to organise ourselves against the harm.

Or corporations: https://www.zylstra.org/blog/2019/06/our-ai-overlords-are-al...

Corporations aren't AIs, they aren't as powerful as AIs, and they don't think like AIs. I have mathematical proof. Show me a corporation that, as a whole, satisfies both invulnerability to dutch book attacks and has a fully total ordered VNM compliant utility function.

That merely makes them stupid AIs.

I guess I failed to understand the point. What I mean is that arguing that AIs can't be a problem (something that I'd like to be true, but probably isn't) because companies already are superhuman does not make sense, for some pretty simple mathematical reasons.

The point is a philosophical argument about what constitutes a powerful non-human agent. Nobody is arguing that corporations are literal thinking computers.

> arguing that AIs can't be a problem ... because companies already are superhuman

Quite the opposite, actually: corporations can potentially be very destructive "paperclip optimizers".

What makes you think that AIs would be VNM rational?

They should either be VNM rational or have surpassed VNM rationality. Anything else is leaving utils on the table (though I suppose that's kind of tautological).

Reply on news.ycombinator.com