If you have some service provided by government, then it's sometimes inefficient. These inefficiencies can only be reduced by better governance, i.e. people actually getting involved politically and forcing the management to straighten it out.
If you have some service provided by only private companies for profits, then perhaps the production is efficient, but this efficiency can become lost for the end customer in the form of profits. In theory, this shouldn't happen due to competition but in practice it happens a lot (almost no private company operates at marginal costs).
So if you have a choice of both options, in theory you should get optimal results. In practice, it is difficult for societies to keep both options, for reasons:
1. If the public option is too good (too efficient), no private companies want to compete (no profit). For example, look at some state-owned European companies, such as Swiss post or railways.
2. If the private option is too good (only decent profit), then there is no reason to have the public option. People want to avoid governance, if possible. The people who pay for private option will question if they actually need to fund the public option from taxes if the private ones are comparably good.
So I think the big problem is that system with both (the best one) is unstable.