Publishers are "content curators", they assure the gamer/buyer that the games for sale are high-quality.
If you flood the market with lots of indie games (with a high ratio of low quality games), review websites will become stronger and more relevant in the industry and assume the role of content curators.
Today, review websites are already powerful. If publishers 'left' the game industry, their influence would be transferred to review sites and indie developers will be in the same position that they are today.