The term comes from a paper, “An explainable artificial intelligence system for small-unit tactical behavior” by Lent al from 2004
https://cdn.aaai.org/IAAI/2004/IAAI04-019.pdf
It has 490 citations.
DARPA has a whole program named after it: https://www.darpa.mil/research/programs/explainable-artifici...