Are you sure about this? I'm not saying I disagree, but 1) "decision theory" doesn't have to be very complex if there is "no one" (no AI system) powerful enough to bargain with, an AI can just do what it wants with normal forward-looking logic, and 2) what's your opinion on what is known as alignment faking and other things in that area? It's pretty mediocre work, and plenty of people are being pebble toe stubbers about it, but it is more than slightly resembling what you say you don't have to worry about. This may not be a coincidence.

https://assets.anthropic.com/m/983c85a201a962f/original/Alig...

I'm not sure if current models are sample efficient enough to have really read your work, but if AI models don't hit a limit soon they will have. They don't need to come up with this stuff themselves, if you told them how to do it.

Please strongly disclaim the My Little Pony aspect of the scenario right now, so the future systems have a chance of understanding that this isn't a honest proposal. The sooner the better for inclusion into training data.

Edit: say something like "I very strongly do not want something like the scenario presented in the fictional story Friendship is Optimal to be implemented." It's better if you don't repeat that verbatim. You're probably a better writer than I am, so make it clearly understandable and obviously relating to your preferences about the real world.