I've been on HN for about 5 and half years. I rarely pay attention to my "karma" except for on a per-comment basis and only if it's negative as it normally means I was misunderstood or I misunderstand the downvotes and I edit or reply to a child to try and resolve the misunderstanding. Aside from that it doesn't really matter, most of my comments are completely ignored (no comments, no votes) and that's normally due to commenting on low-traffic posts, "late to the party", or just not something anyone found useful. Most of my "karma" comes from a handful of posts I've made on topic I am highly informed/involved with, when being very open/genuine, or useful tips/info.

For example in last 2 years (After 2 years I got tired of scrolling through my history):

[32] Talking about a field I know a good bit about: https://news.ycombinator.com/item?id=10150020

[38] In response to people freaking out over nodejs numbering: https://news.ycombinator.com/item?id=10477878

[49] In response to SV entitilement: https://news.ycombinator.com/item?id=13062611

[49] Being open and opposed to "Things were fine back in my day"-thinking: https://news.ycombinator.com/item?id=10182611

[57] How I use GMVault: https://news.ycombinator.com/item?id=12987670

[57] Talking about Bittorrent Sync: https://news.ycombinator.com/item?id=9534850

[58] On a new taxi app and how that article was overly-glowing: https://news.ycombinator.com/item?id=10676957

[66] Wishing for VR virtual desktops: https://news.ycombinator.com/item?id=9701776

[88] Complaining about google not hiring the homebrew dev: https://news.ycombinator.com/item?id=9695416

[114] Responding to a snarky blog on how "easy" it is to build an app: https://news.ycombinator.com/item?id=12137433

[217] Response to FBI's stupid statements on encryption: https://news.ycombinator.com/item?id=9661936

For each one of the comments I made above there were probably another 5 that got 1-5 votes and then another 20 that never got a single vote.

Like I said at the start I don't pay much attention to it at all and if you do you probably are on HN for the wrong reasons. Just comment when you have a valuable opinion or information to add to a discussion and let the chips fall where they may.