Make AI Robust and Trustworthy with CAPSA!

Modern AI models show great potential across various applications, but their deployment in everyday life is limited due to a lack of trustworthiness. While accuracy is crucial, AI models must also recognize when they can and cannot be trusted to make decisions, especially in safety-critical systems. To bridge this gap, it’s essential to develop AI models with built-in trust mechanisms for reliable decision-making in real-world scenarios.

Trustworthiness in AI models can be improved by addressing three risk sources: Representation Bias, Epistemic Uncertainty, and Aleatoric Uncertainty.

- Representation Bias refers to the potential for the model to favor certain groups or types of data over others, leading to inaccuracies in its predictions with under-represented data.

- Epistemic Uncertainty, also known as Model Uncertainty, describes the uncertainty associated with the model’s ability to make accurate predictions based on the data it has been trained on. Epistemic uncertainty can be improved by training the model longer, or picking a model architecture with higher predictive capacity.

- Aleatoric Uncertainty, also known as Data Uncertainty, refers to the inherent noise or unpredictability in the data itself. This type of uncertainty can arise due to factors such as measurement errors, labeling errors, or natural variations in the data. This can only be improved by improving the data source, or manually fixing the inherent issues that lie within the dataset.

To address this issue of AI trust and gain knowledge of the risk metrics mentioned above, we are open-sourcing CAPSA -- a tool that automates the creation of robust and trustworthy neural networks! It is a Python library that utilizes wrappers to make tensorflow/keras models risk-aware. These wrappers work by augmenting a given model to support the risk metric the wrapper provides. The wrapped model gains risk awareness capabilities, outputting risk metrics mentioned above alongside its predictions. Since these wrapped models are simply augmented models, they can be further trained with Keras API.

Checkout CAPSA and STAR our repo if you find it cool or helpful for your projects!

We also have a paper published if you'd like to learn more about the details of how some of our wrappers work: https://themisai.io/papers/capsa.pdf

Let us know what other features you would like CAPSA to support and we'll work on adding them as well!