These days there's an "unvote" text link added next to their name, or "undown" for the case of a downvote.
In addition, on my mobile, when I accidentally or intentionally hit the downvote, it doesn't take the first couple of times. This has saved me numerous unintentional downvotes. Thanks, HN!