The main problem is that upvotes and downvotes are devalued because they are essentially unlimited. What if karma behaved more as a currency and the costs increased as you used it repeatedly in the same comment thread. First one is free, then it costs you 1, then 5, etc.