Oh come on now - don't pretend that comment was some neutral statement of fact. It was a clear implication of moral failure in your opponent's argument. And it's not really persuasive to then double down on a false dilemma (is my out-group evil, or just stupid?).