That seems a ludicrous suggestion. I for one don't read HN every hour, why should I be prevented from downvoting something just because I'm not an obsessive reader?

And any suggestions such as

    "You can only down-vote for up to two hours after
    your first activity that follows a pause in your
    activity of up to three hours, unless ..."

are too baroque to be worth considering.

So ask - why are people down-voting you? Are you saying bad things? Are you saying things of no value? Are you saying things of value and people don't appreciate how insightful you really are?

If you think it's the latter, perhaps you need to learn how to express yourself clearly for your audience. There was a discussion very like this elsewhere recently, and here's something I wrote:

http://news.ycombinator.com/item?id=1021504

The more direct response to the OP would be to limit downvoting based on whether a user has already downvoted another post of the same user already. But even that gets silly.