Have a look at how the shipping container took over global logistics. They also had to redesign nearly everything about that industry, but the savings and efficiency gains were too juice to ignore.
> The point is there's a million of these edge cases for which autonomy will not economically compete for another hundred years.
My prediction: anything a human can do by just manipulating the controls of the car (steering wheel, gas, breaks) will be automated fairly quickly. Especially stuff like parking maneuvers (I assume that's what you describe?), because those happen at low speeds and mostly involve fixed non-moving obstacles. And it's pretty easy to generate training data. When the autonomous system attempts these edge cases, they can simulate a myriad attempts with the on-board computer. There's not a lot of physics going on at low speeds. (Or rather, pretty simple approximations are enough.)
The harder parts are dealing with moving and ambiguous objects (pedestrians, other cars, cyclists etc) at speed. But the industry is well aware of that.
About what will take longer: at the moment a truck driver provides a lot of incidental labour around the truck. Like helping with loading and unloading, minor repairs, etc.
If you have no person on board, you need to redesign how the industry is organised. If you need human help to unload at the destination, you need to preposition a human there, you can't rely on the truck coming with a human.