I think tagging your upvote/downvote with metadata describing WHY you voted that way is not complicated.
Upvote - important
Upvote - funny
Upvote - informative
Upvote - consensus
Downvote - spam
Downvote - rude
Downvote - inaccurate
Downvote - disagree
Randomly assigned metamoderation as civic duty is not too much to ask of readers and contributors.
I also think the complication should reside in the algorithm and not be handed to the userbase. If metamoderation algorithms can figure out which moderators I agree and disagree with, let me favor the data those moderators i prefer contribute to the system.
A well written algorithm will be able to figure out when a person is polluting the system with noise and trying to watch the world burn. Disagreeing with consensus moderation is different than being disruptive. That is a challenging and complicated prospect, but that complexity lies on the backend, not the user facing interface.