Peer review is necessary and time-consuming, that is true. But I do not believe that requires the need for profit-focused enterprises like Elsevier or the other publishing houses. I don't think any field has come up with a perfect solution yet, but as a computer scientist, things seem less grim than in many other disciplines of science.
I publish most of my research in the proceedings of the ACM conferences. ACM is a professional association, not a for-profit enterprise. Peer review is carried out by unpaid program committees, where I believe the primary motivation to participate is a combination of moral obligation, networking (with the other committee members), and an opportunity to get exposed to cutting-edge research. This part is similar to other fields, I believe. The main difference is that paper authors are expected to provide camera-ready PDFs of their paper, with the ACM doing little apart from proving LaTeX templates, concatenating the PDFs, and perhaps arranging a few print runs (but I think it's been a while since I have seen physical proceedings).
Most ACM conferences (and journals) are not open access, but some are. Generally it takes a subscription to their Digital Library to download the papers. My impression is that this subscription is cheaper than for the large publishing houses, but I don't know. In any case, an author can pick various licensing models for their paper, with one of them permitting the author to provide a "preprint" of the paper on their personal or institutional website. In practice, that means the papers are publicly available with minor effort, only that ACM does not take responsibility for the hosting.
The entire process appears quite lean, and the expenses incurred by ACM are covered through general membership and conference attendance charges. Further, the ACM appears to continually (and rather aggressively) move towards even more open access-ish policies, likely because it is ultimately run by academics.
The main difference from arXiv is that pre-publication peer review does actually take place.