First, APCs have not been calculated with regard to the costs of publishing, but rather with the goal of maintaining the same level of profits for publishers' shareholders.
Second, I know what I said is possible because some journals already do it, and there is no scale issues at all here. Journal management is entirely decentralized. One journal = one editorial board.
Let me illustrate this with one of the possible models which is called "overlay journals".
In an overlay journal, papers are uploaded for example to arXiv, and submitted for review by giving the link to the paper on arXiv. Then the editorial board (composed of researchers) does its job as it would in a traditional journal: it sends the papers to review to other researchers of the relevant field (well, at least they try). The reviewers also do their job as they would in traditional publishing. And papers are accepted as is, with modifications, or rejected, again, as usual.
Now what constitutes an issue of the journal is a list of links to the accepted versions (repositories such as arXiv can keep track of the successive versions of a paper) of the paper in arXiv.
There, same level of scientific requirements, no publishers, literaly no costs for anyone except arXiv (which is already paid for by public institutions), and researchers time, that is already spent and not paid for in the traditional system.
If you really want double-blind peer reviews you can use one of the open journal systems free software or as people currently do, public services such as easychair or scienceconf to organize the reviewing phase.
The only thing that is still necessary for some fields, as I already said, is the proper layout work. The ideal situation is that each university or group of university hires a few professional in charge of this work and who can do it in close contact with the authors of the paper. A few salaries costs less than the millions currently spend each year by each university for journal subscriptions, and the job would be done better than it currently is by publishers who often outsource it to low-salary regions of the world to increase their profits.