Combined with the idea that folks think models would be better used if they presented their uncertainty, I can see the direct line to models needing explainability before we deploy them.
To that end, why do you think "no?"
As I understand it, uncertainty is a statement of risk. Explainability is statement of understanding how a system works to produce an outcome. None of the four NIST principles seem to conflate the two.
I can say I understand how my brakes on my car may fail to work because it's an explainable mechanical system with known failure modes. However, that's different than the statement about the uncertainty that the brakes will work as intended. In the latter, there is a statistical probability that gets translated to a risk statement. I think one needs to have an explainable system in order to arrive at an uncertainty risk statement. They are both related to quality, but speak to different aspects of the problem.
That is, yes, they do ultimately tell you different things. But, per the article, both can be used to push back on using a model.
That is to say, in prior years, folks pushed back on models for them not presenting their uncertainty. Seems there is a growing push to push back if they do not present explainable reasons.
IMO, if explainability is the new way to push back on models we're uncomfortable with, it shouldn't be. Uncertainty arguments can be mathematically quantified and defended. Can the same be said for explainability? (Genuinely asking). If not, it's really just a less rigorous way of saying "I'm not comfortable with this model but I can't explain why."
As an example, you are treating uncertainty as a form of tolerance. But you have to explain that, as well. Why is one model 10% uncertain, but another is 30%?
You could just say it is over the data that was trained, but if you can pull it back to used parameters of a model, they may make something more obvious. And it is hard to take uncertainty based on trained data something that transfers to unseen data.
Yes, any model is limited by the data used to build it. Relying on that isn’t particularly helpful, just like saying there are unknown unknowns, while true, isn’t helpful. What helps regarding uncertainty, however, is that it can be explicitly defined. Defining uncertainty in parameters is part of the effort; the parameters can be defined within uncertainty as well rather than assuming a point estimated parameter is gospel. That’s one way that helps explain why one model has different uncertainty than another. Some statistical methods, like Bayesian inference, require you to define these assumptions mathematically. All models require assumptions but there’s a world of difference between a black box and one that requires explicitly and mathematically defining them.