We all know that the fastest and most reliable way to find and fix bugs is to implement the thing in production and see what happens. As you point out, it exposes problems that testing and design just can't.
I see people here obsessing over how much will change if we had fleets of AVs. Maybe shaving five years off the wait time is worth N number of human lives. Especially if N < the number of lives will ultimately be saved if the technology is perfected years earlier.
Of course I wouldn't want myself or my loved ones to make that sacrifice. I don't want to die in a regular car accident either. Given the upside, speeding up AV development may be worth the risk?