Why are these intermediaries still around? Peer review is the real value provided by a scientitic journal and the people who do that have no reason to be loyal to these companies since they don't get paid.
Because publishing a decent journal - even online but certainly in the real world - involves a whole lot of work: Administrative, some technical, some networking, some advertizing, maintaining relations within the relevant fields etc. That's all when we ignore the peer review itself which requires domain expertise.
Taking into account that they don't really do peer-review and don't pay for it either, and that their editing is close to nonexistent (and as likely to introduce errors as to improve anything), all the other costs seem to be a self-perpetuating loop - the work is being done just to support itself, with no actual surplus value being produced. It's a resource leak, a circular reference in the economy.
It's not meaningless, either. If you have a hundred venues for publication - what should you choose to follow? What should a library subscribe to? What do you recommend to students? etc.
I am also a big fan of open-access and hate paywalls myself. I also don't like big, for profit, predatory journals. There is value in curated and managed journals. Just not the value that these big corporations are reaping from them.
The others require the appearance of having done so.
Out of curiosity: why don't you just host your papers on your website then - or, if you do, why don't you think it's enough?
Most publishers prevented you from doing this as of a few years ago (I left academic research around that time). You were not allowed to post any content that contains any work by the publisher (even formatting/editing changes after first round of reviews). Thus you could only publish a "preprint" which no-one can rely on to cite because they don't know what's in the final peer-reviewed version. Some publishers are more lenient than others but there's definitely friction induced.
(And ideally this note would also mention any substantial changes, like "new section 4 explaining..." or "derivation in 3.2 re-written, but no change to conclusions", for the benefit of those who already read v2.)
As for why it's not enough—because hiring committees and funding agencies almost never take unpublished (that is, in a proper journal) manuscripts into account when evaluating you and your funding proposals. This is what needs to change in the first place to break the loop.
Which country are you from? Where are you now, in industry? Is there less toxic environment than in academia?
I’ve seen plenty of researchers say “information should be free” and then later publish everything in closed journals because the open access ones have low impact factor.
Seems hypocritical to me.
If so, why is the prestigious curation of the journal that leads academics to reading your paper and taking it seriously inextricably tied to whether or not it's behind a paywall at all?
What's stopping someone from building an alternative curation pipeline on top of open access journals that gives academics equal or better signal/noise to the journals they read, and a similar socially accepted prestige for getting into that curation pipeline?
I get that there's not a clear path to monetization, but maybe it's possible, particularly if you could execute more targeted curation for academic subfields that are too small to have their own journals, that you could find some donors and lean on academics supporting the curation process themselves out of their seeming discontent with publishing to drive down costs.
A lot of scientists talk big when it comes to open access, but when it comes time for them to publish their own work, they published in closed journals since they have a bigger impact factor.
They basically put their own careers ahead of their belief in supporting open access.
That is a substantial sum even for professors from wealthy countries.