Social media upvoting/downvoting that results in comments being hidden is the reason for this. Even a small imbalance in one direction or the other effectively silences one side. Perfectly legitimate, well-reasoned comments will get angrily downvoted into invisibility. Echo chambers get created and enforced. This is particularly bad on Reddit (low karma can get you banned from a sub) but even on HN I'll give anecdotes that go against the "party line" and I'll get downvoted.

Problem is, this is the best way to increase engagement because it gives the most people the most compelling content. So this will never change.