karma at hn is broken. ignore it.
some quick example, taken these from my own history just because that's what i know (but you can see examples everywhere):
- voted down for no reason (it was a useful, informed comment) and no explanation - http://news.ycombinator.com/item?id=1190927
- receiving no votes despite being (closest to) the correct answer; a nerdgasm of technically correct but irrelevant info gets 9 votes - http://news.ycombinator.com/item?id=1194538
voting helps introduce "real world" social pressure into an online community, but here the feedback loop is way too strong. the result is an echo chamber, because people like affirmation, and because it's easier to vote on whether items "state the group consensus" rather than actual useful content.
things might be fixed by not displaying karma, or not displaying vote counts, or making voting "cost" karma, or similar, but the people who run the site don't have the time or inclination to experiment.
in short, as i said, the best thing to do is ignore karma.