There is also a UI/UX problem on mobile, the buttons can easily be fat fingered. There'd be a nice externality to this extra step.
But thanks for "undown"/"unvote" links (which are bigger and thus easier to hit) this has not been a big problem so far.