Paying a guy to sit in the robovan and run the packages to the door isn't going to be appreciably cheaper just because he sits in the passenger seat.
There aren't any technical unknowns left. Obviously the algorithms will continue to be tweaked and improved. Importantly it can operate in the existing environment - there is no need for every car to switch to driverless at once.
Remember that anyone locally would also need to spend a similar order of magnitude in fuel (and likely more) if they went to get things themselves.
I don't think people have fully thought through the impact of this yet.
A year ago I dared to suggest on HN that self-driving cars could potentially cause congestion (because it would be cheaper for an electric car to sit stationary traffic rather than paying for a park).
Back then I was downvoted to oblivion. I hope now people think it through a bit more.
I reckon self driving cars will also lead to significantly less car ownership. Taxis will become cheaper, more flexible and more plentiful (political shenanigans aside). The Zipcar model works too as the car can come to you.
And of course the cars can be coordinated to alleviate congestion. For example the system may work out that a particular car going at 29mph for a stretch instead of the limit of 30mph would avoid a glut of traffic 5 minutes ahead, or take a different route, or tell it to wait 5 minutes when it is empty.