Academic communities are often quite small; back when I was publishing and serving on committees it often wasn't hard to figure out who the reviewers were, because a paper proposing a revolutionary new way to do X would be sent to top world experts on X. In those cases, the fact that you could sometimes figure out who was who limited possible bad behavior, because clearly unfair reviews would damage the reviewer's reputation.