So, there's an area of research that's under way called "AI Assurance" which seeks to answer many of these questions.
Some things they're attempting:
- Creating explainable outcomes by tracing the inner works of ML models.
- Looking for biases in models using random inputs & looking for biased outputs.
- Using training sets with differently weighted models to find attacks and biases.
etc.