I totally agree with you.
So why hasn't this happened yet?
I believe this is because what we have here is not a technical problem, but primarily a regulatory and legal problem. If the political leadership of the NHTSA had the will to make highway self-driving happen, they could publish a draft safety and liability standard.
The technical problem here is that while shuttle and robotaxi systems can go slow enough as to make fatal accidents rare, as you said, the most consumer benefit potential is for a highway self-driving system that would not require active monitoring like Autopilot does.
The political/legal question is: when fatal accidents occur, who has the legal liability? If it's the manufacturer, then why would they accept the risk? If it's the driver, then who would actually go to sleep and let their robot potentially kill people? Especially given the problems that Tesla autopilot has with stationary targets in the road.
It's possible that the key innovation required here is not anything technical, but an insurance product that would allow the manufacturer to assume the liability, but be insured for damages. This insurance product would involve a deep technical evaluation of the self-driving system to get actuarial-level probabilities of mortality.
On the NHTSA website, there are many >50 self-driving test sites, but only 3 which are classified as "highway". The most common vehicle type is "shuttle". A full self-driving airport shuttle, with purpose-built road markings, makes a lot of sense as a goal achievable with today's technology, and is far more likely to kill people due to its controlled built environment and slow speeds.
https://www.nhtsa.gov/automated-vehicles-safety/av-test-init...