The drivers in Uber handle more than just the driving - they essentially handle all the other logistics of offering a taxi service. They keep the car clean, they handle any regular maintenance and servicing, they store the car when it's not in use, they manage registration and insurance, they manage any insurance claims, they fix any damage, they mop up the vomit, etc.
Once Uber goes self driving, they take on responsibility for all of these logistical activities, and they currently have no capability and capacity to do that. These activities only scale so far.