This is very perceptive. Attempts at honest discussion often seem to be derailed in exactly this way. Discussion of the actual issue is avoided by loud assertions that take a lot of time to refute. Which "side" derails it does seem to depend on who the "weak man" would hurt.
Is there any way to avoid this?