Drivers also can't dispute their ratings so it takes one person to give them a one star which could lead to being fired. And can you truly say all customers are fair in their ratings?
The best thing that could happen for these drivers is having another company open up to compete against uber. Then they wouldn't be a commodity.
If they were not "taxi drivers" than everyone would be furious at how they are treated.