You've posed some interesting questions.

> Then again, if an AI system can't understand how itself works, is it really intelligent?

This question is particularly interesting as it can be applied to us as well.