How can we be afraid of AGI if we can't define consciousness?
youtube.com
youtube.com
Bernard: “Life like, but not alive? Pain only exists in the mind. It’s always imagined. So what’s the difference between my pain and yours, between you and me?”
Ford: “This was the very question that consumed Arnold. Filled him with guilt. Eventually drove him mad. The answer always seemed obvious to me. There is no threshold that makes us greater than the sum of our parts. No inflection point in which we become fully alive. We can’t define consciousness because consciousness does not exist. Humans fancy that there’s something special about the way we perceive the world, and yet we live in loops as tight and as closed as the hosts do. Seldom questioning our choices, content, for the most part, to be told what to do next. No my friend, you’re not missing anything at all."
TBH I am more afraid of a "fake" AGI than an actual AGI, as the pseudo AGI is more prone to irrational logic chains that end in scary outcomes.
Saying "consciousness is an illusion" makes no sense except for twisted definitions of "consciousness" or "illusion".
Most people synonymize "consciousness" with "subjective experience" and "illusion" with "not real". So if you're saying "consciousness is an illusion", you're saying consciousness doesn't exist / that no one has subjective experience / that no one can observe anything / that everyone is a philosophical zombie.
Surely you can prove to yourself that you are observing something. Maybe you're confused about what you're observing, but you can be 100% sure you're observing something. In fact, all of your knowledge might be an illusion, except for one thing: you know that you are observing/experiencing/conscious. It is the ultimate non-illusion. Another way to put it is that this is, by definition, what it means to be conscious.
"I think, therefore I am."