There's a rebuttal to this as well: Attention is not not Explanation. https://arxiv.org/abs/1908.04626

Thanks!