A good journal needs good reviewers.

The quality of peer-review at conferences for example is terrible, and that is well known. Even at not very big conferences, and despite the usually hard work of the organizers.

So establishing a good review process is not easy and perhaps that is what is stopping the open access model rather than just the prestige issue mentioned in other comments.