Open access journals have failed in part because they lack the prestige of top tier journals like cell or nature
The reason traditional publishing has failed is because even at the highest tier universities there is not universal access to content or even access to the majority of content. Sure my University will buy me whatever paper I want, but I don't want to wait a day to get a paper and then discover that it is not actually pertinent to my work when I can get it on scihub in five minutes without having to fill out a request form. Until there are bundled services a la Spotify which provide access to almost everything, scihub will continue to flourish.
The issue has very little to do with price and much more to do with convenience to the researcher.
It's interesting how the journals managed to get a monopoly on this over 50 years. There seems to little reason for a private entity to be able to inject themselves here and take control of public output.
It's like the public paying for oil exploration with private companies like Exxon Mobil stepping in when it is found to take control and sell the oil. This doesn't seem to have anything to do with capitalism or any ism, it looks like pure theft.
I think it's symbiotic relationship between universities. Universities get together and decide who to pay to distribute their "best" research, in effect creating a prestige/walled garden affect surrounding academic publishing, raising the barrier for non well monied researchers having access to the audience.
If universities decided that they would no longer publish in nature et al., and just put publications on arvix or something, then the monopoly would effectively die, but It's not in their interest to do so since universities will have to compete on mind-share on what papers attract the most interest in a field and chip away at the advantages they have today.
Long term though, universities will loose their advantage as it becomes easier to engage and propagate knowledge outside of such.
The value is in the brand. The brand value includes editing and keeping up the quality.
With open access model, high prestige journal could ask much more per publication than $1,500 - 3,000 normal open access journal does.
If they the cost of publishing in Nature would be $15,000 and they would continue publishing 800 articles per year, it would make $12 million per year. If they would double the number of articles published they could make almost the same money with smaller fee.
New open publication journal gets no reputation, so nobody reads it, and nobody submits to it, so it gets nothing, and can't ramp up its reputation.
Network effect.
The classic network effect example is the telephone - the system is is of zero utility if no-one you know uses the system.
Reputation effects are more decoupled and fuzzy. A tiny niche journal with small readership can nonetheless have a high reputation if it only publishes highly regarded articles.