On the other hand, if you place Magnus Carlsen against AlphaZero in a game of chess, I will bet on AlphaZero. If however you reduce the complexity of AlphaZero down to a level that it can produce an explanation I can understand, I would instead bet on Magnus Carlsen.
Of course we should care about the quality of AI systems, but chasing a human understandable explanation is just the wrong way to go about it, since it in many cases necessarily limits quality of the decisions.