I truly don't understand why academics put up with journals. If I were in the biz, I'd be organizing a torches-and-pitchforks mob to burn them down and replace them with arXiv, personal websites, and peer review panels who link to good papers.
But it is also a realization that this isn't just a shallow local minimum. The lifestyle and security of many researcher are tightly coupled to getting funding out of this system, and thus is tightly coupled to following the rules of the system. Either some external force is going to have to shove this system aside, or the members of the community are going to have to collectively accept the pain of a governing body seeing the writing on the wall and attempting to regain control. With the system as entrenched as this, it would have to be a coup of epic proportions to unseat it. The only other option I can see is this system remaining in perpetuity.
I noted above that being in a similar position, I'd probably work this system the same way most everyone else does. I might provide the usual excuses as well: family, food, survival, lifestyle, etc. I don't know, because normally it takes a person being put under considerable pressure before they decide what is truly valuable to them, and to what degree they are willing to sacrifice one goal for another.
I took a third option while going through my undergraduate years; I decided to not even get involved. I am not saying this as a sanctimonious thumbing-of-my-nose at those who did; these people have my respect for being able to work around the flaws and not be driven mad by them. The allure of being able to work on interesting problems with smart people tempted me toward working to get involved in research, but the glimpses I caught of the politics, mechanisms and incentives chased me away. My significant other at the time was involved in a research effort in computer vision, and the amount of inanity she tolerated, and even in some cases manufactured, killed the desire for me.
Since then, I've been told by a number of people I have the mind and personality for research work. Even if that were true, I know I don't have the personality for all the bullshit that comes with it. My philosophy on this is the same that jwz suggests for people thinking of working at a public company: "sometimes the only way to win is not to play."
Everyone who replied me seems to have missed that. Basically, review panels become journals turned inside out. They don't host the content, but they do certify it.
If you're going to go through all the effort required to peer-review and certify work, why wouldn't you also throw up a website and collect the papers you certify?
Why would actual science come in neat magazine-sized packets? It's obvious they must be normalizing the signal, either by floating the cutoff or just picking until the slots are full and trashing the rest.
A review panel that doesn't have to pay some company to stamp ink onto wood pulp can afford to link as many, or as few good papers as arrive. And, they can afford to do this asynchronously, as soon as they arrive.
Experts in your area---those who can actually understand your paper---mostly won't care whether your paper's been peer-reviewed or is just on the arxiv. You get it published mainly for folks who can't (or don't want to have to) understand it. You put up with all the crap this entails because the latter category usually includes people responsible for hiring you, promoting you or awarding you grants.
Those random factors don't even have to be nefarious; an expert in an area may not have sufficient sub-domain knowledge to fully understand the work of another author. The hope is that the publication is clear enough that they can, but every publication has to make a trade-off about how much knowledge they assume the reader has.
Each of these random factors chisel away at the intended meaning of the certification. If enough of these accrete, you end up with something that tells as much about a potential publication as a GPA does about a high school student. But then, people depend on things like that for hiring and promotions as well. It would be a damn shame if this is really sufficiently optimal that everyone stops working for better.
Yes, it is true that I've read a good paper that I do not care whether it is peer-reviewed, on arxiv, or just on a homepage. As long as it is of good quality and there is a way to cite it, I'll probably cite it in one of my own articles.
However, and I'm not saying this out of defense for the current system, you must understand that the system is also in place to filter out some of the complete and total crap some people dare to call research. I've reviewed articles that made my toes curl, because of bad grammer, bad presentation, or just duplicating another article with 10% of the article changed.
Already at the moment there are so many journals and magazines that I have to keep track of that I simply don't have the time to search the web or dig up things on arxiv. I simply have to trust some system to filter out the good articles.
The problem is that the whole system is fundamentelly broken and it is going to take a complete mindshift of everyone involved to change that.
The OpenAccess movement is something that may have a chance now that funders are also seeing the positive effects of it. But this is going to take years to change.