That's a possible side-effect. But it has benefits (for e.g. better content discovery) that I think out-weighs the side-effect.

On the other note, if vote count is the measure of content quality, the ease to vote should not adversely affect the system but improve it.