Not all ML is built on neural nets. Genetic programming and symbolic regression is fun because the resulting model is just code, and software devs know how to read code.
At least with symbolic regression you can treat the model as an analyzable entity from first principles theories. But that's not really particularly relevant to most failure modes in practice, which usually boil down to either missing some qualitative change such as a bifurcation or else just parameters being off by a bit. Or a little bit of A and a little bit of B.