Since you mention it...
I recently lost 11 points in the span of a few minutes (while writing a comment), apparently because someone went and downvoted all my recent comments.
I've seen other suspicious behavior for months, but it's more anecdotal, so I won't go into it here.
tl;dr I am probably getting vote brigaded.
edit: And now I am "submitting too fast" and can't continue the conversations I am in (though I can still make edits). I don't know how the algorithm for that works, but I suspect (from experience) it's partially based partially on how many downvotes you're getting, because I often hit this without submitting quickly or all that much. Also from experience, it takes a really long time before you can start submitting again. So I'm effectively silenced (until next time...).
edit2: "If you talk about downvotes, chances are you'll get downvoted" (below---I'm still "rate" limited). Well, someone else raised the issue of culture on HN (not me), and I'm providing data. This is constructive, as opposed to simply bitching about getting downvotes, which I am being accused of. That is wrong and I never do that, as my comment history will prove. This accusation is conflating two entirely different things and is really unjust and unhelpful. Again, having someone systematically downvote your comment history is not the same as complaining about people downvoting a particular comment because they don't like it. Calling me a "bore" because I am reporting abusive behavior is beyond the pale.