The primary reason we use large buses rather than e.g. minivans is that it amortizes the cost of the driver over more people. If it weren't for that buses would be less efficient because they're very often not completely full and to be completely full in most cases requires a larger vehicle to travel more miles.
What you really want is, effectively, car pools. Someone was already going there and they just take on more passengers, so there is no separate driver you have to pay.
But Uber already has this. It's called "driver destinations." You tell it you're driving to work, it gives you passengers near you who want to go near there. You make some extra money on a trip you were making anyway and now your car has five people in it instead of one.
Is the problem just that people don't know this?