This is an advantage when learning but also a disadvantage until that learning is done. Which is why beyond Tesla's intentionally misleading branding for the AP I consider the drivers are ultimately responsible for controlling the vehicle.
I like to bring up an example of a radiotherapy machines here - a machine that treats you with radiation cannot have a state in which it kills the patient, even if the chance of that happening is 1 out of a million. In that scenario it doesn't matter that a manually operated machine would kill more people on average - an automatic one should kill absolute zero.
Or maybe for another example - plane autopilot has saved countless lives. But any time one fails, every single plane of that type is removed from service until the issue is found and fixed.
I feel like at the moment the approach these companies take is "well yes the autopilot can make mistakes, but it's still safer than manual driving so it's fine, yeah?". No tesla, it's absolutely not fine on any possible layer. Tesla should be disabling autopilot on every single Model S and Model 3 sold until the issue is found and fixed.
>>It's on consumers to understand this difference and pay full attention while driving.
Then this is a crazy assumption to be making. Like literally crazy. Pilots are trained over and over again to pay attention - and they still fall asleep during flights. With cars, even engineers who are employed and paid(!!!) to pay attention have fallen asleep while testing level 3 autonomy[0]. Expecting a regular customer to both understand the difference and pay attention is wishful thinking. The lowest bar for this technology that we should be aiming for is "the vehicle should never under any circumstances fail to observe and react to a stationary object in front of itself". Simple as that. If Tesla's autopilot cannot meet this bar it shouldn't be on the road, no matter how many times safer on average it is than a regular driver.
[0] https://www.thedrive.com/tech/7730/ford-engineers-are-fallin...
What's extremely worrisome is that multiple people have died from the exact same failure in Autopilot, indicating that Tesla's claims of "data-driven" learning are bunk.