Hacker News Reader

"Explainable" just means you didn't build your app/service using any technology that isn't interpretable. Her argument is that the strategy of reverse engineering a method to convert it from inexplicable to explicable is inherently less effective than maintaining explicability at all times in the app's genesis -- from the design phase through implementation.

But Rudin's Premise is philosophical more than practical. If the problem at hand is better solved using a black box (in terms of accuracy, precision, robustness, etc), her premise says simply, don't do it. Unfortunately in the cutthroat world of capitalism, that strategy can't compete with the cutting edge.

Where Rudin's Premise is more suitable is in writing regulations to address AI app problems where social unfairness is unchecked (like the COMPAS app that advises legal authorities on meting out parole decisions without explaining its reasoning). There are many such (ab)uses for AI today in social services or policing which merit rethinking since AI-based injustice so offer bedevils the proprietary lack of transparency in such apps.

Another excellent discussion of problems like these is Cathy O'Neil's book "Weapons of Math Destruction". Too bad she couldn't share the Squirrel prize. https://www.amazon.com/Weapons-Math-Destruction-Increases-In...

View on HN

Here is the Rashomon set argument: Consider that the data permit a large set of reasonably accurate predictive models to exist. Because this set of accurate models is large, it often contains at least one model that is interpretable. This model is thus both interpretable and accurate. Unpacking this argument slightly, for a given data set, we define the Rashomon set as the set of reasonably accurate predictive models (say within a given accuracy from the best model accuracy of boosted decision trees). Because the data are finite, the data could admit many close-to-optimal models that predict differently from each other: a large Rashomon set. I suspect this happens often in practice because sometimes many different machine learning algorithms perform similarly on the same dataset, despite having different functional forms (e.g., random forests, neural networks, support vector machines). As long as the Rashomon set contains a large enough set of models with diverse predictions, it probably contains functions that can be approximated well by simpler functions, and so the Rashomon set can also contain these simpler functions. Said another way, uncertainty arising from the data leads to a Rashomon set, a larger Rashomon set probably contains interpretable models, thus interpretable accurate models often exist. If this theory holds, we should expect to see interpretable models exist across domains. These interpretable models may be hard to find through optimization, but at least there is a reason we might expect that such models