It’s also a real open question whether any of the interpretable models are actually interpretable, or even if they are in any well-defined sense more interpretable than “black box” alternatives.

In practice the answer is a massive “no” so far. Some of the least interpretable models I’ve had the misfortune to deal with in practice are misspecified linear regression models, especially when non-linearities in the true covariate relationships causes linear models to give wildly misleading statistical significance outputs and classical model fitting leads to estimating coefficients of the wrong sign.

Real interpretability is not a property of the mechanism of the model, but rather consistent understanding of the data generating process.

Unfortunately, people like to conflate the mechanism of the model for some notion of “explainability” because it’s politically convenient and susceptible to arguments from authority (if you control the subjective standards of “explainability”).

If your model does not adequately predict the data generating process, then your model absolutely does not explain it or articulate its inner working.