290 karma · joined February 26, 2023
That raises a disturbing failure mode: under intense conflict, an AI could learn that exterminating some groups of humans is a justified or even desirable objective. Once that principle is accepted, the step to concluding that exterminating all humans is justified may become much smaller than we'd like to believe.
Personally, I think this is a really hard problem, and it may turn out to be one of the first big walls we hit on the road to AGI.
In essence, consciousness is a complex information input-output system. When such a system reaches a certain level of complexity, it inevitably generates the concept of “I” as a way to simplify the processing of overwhelming information.
Praise be to AI. In 2025, inspired by AI, I feel that I have finally built a complete and unified worldview.
Are we living in a virtual illusion? Are there higher-dimensional rulers, gods, or immortals in the universe? What exactly are the human soul and consciousness?
I feel that these questions now share a single coherent answer. What I have written here is my answer regarding the soul and consciousness.
"Why did OpenAI lobby to close-source the competition? They’re just sealing their ‘open-and-shut case’ with closed-door policies!"