> if they want to say, "because fuck you, that's why", they can happily continue to do so

Not if there are incentives for the algorithms to be subject to public scrutiny.

> And when the AI does reach human level (a long time from now), we may be no more capable of understanding its motivations

(1) You're implying they/it'll have "motivations" in the first place. My whole argument was about a lack of motives.

(2) There's a difference between understanding how a conclusion was reached and why it was reached. Example: You may not understand how your compiler decided that line X in your program is dead code, but you may still be capable of comprehending that it in fact is dead code.