Roughly, the number of upvotes on their stories and comments minus the number of downvotes. These don't match up exactly; some votes are dropped to prevent abuse.
Usually, I write a short reply on some thread and although my post shows me "1 point by anthay" my karma doesn't change. I'm sure it used to.
"Please don't comment about the voting on comments. It never does any good, and it makes boring reading."