You've already described the solution to the problem: we upvote things that we think are unfairly downvoted. If you say mildly controversial things, you can often watch this happen on your comments. Not that I wouldn't like reasons attached to downvotes and flags...
ironic that this was flagged an downvoted with no response, ha.
Yeah, that gave me a bit of a laugh too, but the flag seems to be gone now at least!.
Not ironic at all. Downvoting and flagging exist exactly to handle content that is not worth the effort or even harmful to respond to (remember bullshit almost always takes more effort to debunk than to create). As such, it's usually a mistake to both downvote and respond.