All you are doing here is convincing me that tech companies are just runaway trains with nobody at the controls!
Can you explain or understand the algorithms humans use to drive cars?
Explain to me step by step how you walk.
Believe it or not, your car is not that primitive when compared to a self-driving one in terms of the number of things it does autonomously.
If a company makes a self-driving car and that car then drives badly, surely the response needs to be to incentivise the company to improve their engineering practices, eg, spend more on testing, or require more levels of review of changes, or whatever other organisational changes they need to make safer cars. You don't need to find an individual person responsible to create that incentive. And if you really do want to find an individual responsible it can easily just be the executives of the company (and the executives are probably pretty easy to find even a decade later).
Machine learning is very widely used in the sciences and extremely beneficial to humanity in uncountably many ways and assuredly countless more to come. Of course technologies can be used for evil but so can nearly everything that exists. I believe your proposal comes from a desire to help or better the world, but to ban all non-human-readable algorithms is frankly ridiculous and demonstrates a naive understanding of the issue. It sounds a lot like the calls by the U.S. Congress to ban encryption.
- In medical: your doctor should be responsible for your diagnosis and drug company is responsible for defective drugs, except when they get away with lobbying and hiring good lawyers.
- In physics: I'm not sure if it's as big of a problem as in social networks. But consider this case: If you cannot reproduce the result of an experiment due to a ML model being cryptic, that would lead to huge credibility issue in science.
Patients don’t care how cancer is detected. Patients care if the diagnosis is correct.