In your model, a modern day Einstein would have never had a chance; industrialized research, with publication mills and systematic co-authorship permutation is exactly what would flourish under your model.
The only way to game the system would be to somehow convince a ton of highly upvoted people in a given subfield to upvote a paper in that particular subfield (and give up some of their voting power accordingly, since voting would cost you karma). Heck, make it so that you can only vote on a certain number of papers each month (5-10?), to make votes even more valuable.