I build AI/ML systems for a living and it's not as big of a deal as you think. Production machine learning systems built by experienced teams will almost always have business logic protections built-in to stop the machine from doing something totally stupid. For example, an auto-bidder may have capped velocities or auto-shutoff mechanisms (those examples of bidders spending billions of dollars in minutes are exceedingly sloppy).

This is not unlike the human body's deeply-evolved reflexes - before the neocortex can even process that a pan is boiling hot, the nervous system bypasses the brain and and tells the hands to drop it.

In the case of driverless cars, DannyBee is absolutely correct: truly driverless cars will not make decisions from a single data point. I have some past experience with unmanned aerial vehicles and those things had insane amounts of redundancy in almost every signal and decision making process; driverless cars are more complex and will come decades later and will probably take this even further.