More importantly, while you are at work, it pick ups other riders. Which is why people won't need to buy these cars.
The broader issue is that, if this sort of thing were to become commonplace someday, why wouldn't it just become a race to the bottom where your income/mile =~ operating costs per mile. (Or, even more likely as with Uber today for many people, operating costs per mile as calculated by a lot of people who don't really understand their full operating costs.)
Maybe fewer people will buy cars and more will use robo-taxis. But loaning out your car to compete against fleet operations does not sound like it would ever be a good deal.