Being able to explain why automation made a particular decision only has value if people can use that explanation to correct errors. The problem with automation is that there is an extremely strong tendency for companies to just shrug and do nothing, with the excuse that the automation may have screwed up, but that's just how it is.