Downvoting can also mean that the downvoter doesn't think it's worth the time to engage with prose. It's a bit arrogant to assume that a downvote means someone can't come up with a cogent argument against you; the fact that you make that assumption points even more strongly at the idea that it's not worth arguing with you.
I would say that classifying an argument as "silly" without any supporting evidence is borderline ad hominem because it doesn't make a substantive critique of the assertion, and instead colors the character of the opposition. But that shouldn't take away from your position.
As for your suggestion that I am being arrogant, I can see where you're coming from but I also feel as though I'm within the bounds of fairness to push for a reasoned response (instead of an emotional appeal).
All that said, thanks for replying in a way that advances the discourse. Much appreciated.