It's funny that your explanation by analogy of death rests on the assumption the machines already understand how themselves work. There's really no reason why that should be case.
Then again, if an AI system can't understand how itself works, is it really intelligent?
I don't know the answer to any of these questions. I'm just asking them in effort to help myself try to understand technology and AI.
Maybe simple curiosity and the ability to learn and retain data is the key to AI.
Will the first generation of true AI be like a simple human child or a god-like "being"?
Why even bother making pure AI systems when we are so close to BCIs? Why not just take recently deceased people, keep the brain alive, and attach a computer system to it?
> Then again, if an AI system can't understand how itself works, is it really intelligent?
This question is particularly interesting as it can be applied to us as well.