Let's not forget that this whole business would not exist if researchers weren't so addicted to the prestige offered by glamorous journals. Most of the people complaining about scientific publishers are actually complicit in their success and dominance.
The "prestige," as you call it, is necessary to get a job in academia. Where you publish your research is considered a trustworthy approximation to the quality of your research, and requires a lot less time & expertise to vet.
Basically, the same reason why it's easier to get a job offer if you already have a job at a great company: "Someone vetted him for us, he is probably good" - so much easier for those hiring this way than if they'd actually had to check your previous work.
Bingo. Until a few heroes like [that Fields medalist who works with Tao] stood up and took their prestige elsewhere, nearly all academics were happy to let Elsevier run all the journals and make everyone pay. Field-leading academics and institutional administrators chose to hand over control of researchers' careers to Elsevier.