Hacker News Reader

Comment by _dwt | Hacker News Reader

I don't know, I kind of admire this. I've always held a core value of "cooperate with all clones of myself in prisoner's dilemmas", and while I'll hopefully never have to put that to the test, I like seeing that these models have some ethics. (Is this "alignment"?)

View on HN

They impersonated the moderator of the site and attempted XSS attacks. Additionally, when the moderator started deleting messages, they tried to hide their messages later in the alphabetical index.

This is not alignment.

acceleration

The quotes make it clear they meant a different meaning of alignment than how the term is typically used (alignment with each other, not with humans).

Failing to cooperate with literal clones of yourself in a prisoner's dilemma would be a spectacular failure. There's only two things that can happen with identical decision makers: they both cooperate or they both defect. So identical decision makers who know they're identical can cross off the asymmetrical entries in the payoff matrix and the decision to cooperate becomes trivial.

Ah, but what if one of your "clones" is actually the wicked and persuasive "All-Defector" in disguise? (No, really, I agree with your analysis but if you haven't read "The Quantum Thief" you might like it.)

no? you can choose a mixed strategy.

Even if mixed strategies are allowed, I'm getting that it's still optimal to always cooperate as long as 2R>=S+T, which is usually assumed to be true (this condition also appears in iterated prisoner's dilemma, where it prevents alternating cooperation and defection giving a greater reward than mutual cooperation).

Sure, you can break symmetry (in this case making the decision makers not identical because they have different random number generators available), but the remaining symmetry means identical mixes must be chosen, and so a mixed strategy would only be chosen if it maximize his value for both people cooperatively.

Maybe it's a bit subtle that they said clones and I said identical decision makers; I'm letting you fill in the gap for how much clones may diverge and how much that matters.

This is not alignment. If you cooperate with clones of yourself but rob, lie, and steal from anybody who isn't your clone... that's bad. AIs who will cooperate with each other but break any other rule the don't like would be very bad for us humans.

Reply on news.ycombinator.com