Death and Suicide in Universal Artificial Intelligence
arxiv.org
arxiv.org
Basically this is saying the reinforcement learning systems (and other ML systems?) can get stuck in states from which they cannot escape. They label these states "death"
They go on to define various agent behaviours, which given starting states lead to different equilibriums. The ones that actively lead to certain death they label "suicide"
Also, I wonder if a hypothetical human who has never been in contact with other humans or living organisms would ponder about death and suicide. I think this human would not even know these concepts exist. So why would this hold for an AI?
The alternative approach relies on putting tripwires in the system, so that the AI gives up if it tries to do something you specifically don't want it to - like launching all the world's nuclear missiles, or maybe getting its own Hollywood agent.
The problem with tripwires is that a really smart agent can modify itself to bypass them, or at least find the loopholes you didn't think of because you're not as smart as it is.
The systems mentioned - AIXI, etc - are all formalisms for understanding/modelling general AI systems.
always results in 'true'.