To be honest, I think the up/down notion of karma is the problem.
I'd prefer to see words being attached to comments instead of numbers. "Funny", "Insightful", "Accurate", and their opposites could be the method of rating, with "leader boards" showing the funniest, most insightful etc.
The notion of me as a user being "worth" #### points would vanish, and it would be replaced with a general "run4yourlives: slightly funny, not often accurate" descriptor based on the ratings of my peers.
Slashdot tried this, but they kept linking it back to numbers, which I think is where it failed. The idea should be to remove the notion of a numerical worth altogether.