I think this question embeds two assumptions that aren’t constructive.

First of all, it associates the “modern web” with Twitter and Reddit. There is more to modern websites and apps than those two companies. There are a lot of beautiful and useful websites that are built with modern tools.

Second, the question assumes users are hated. Quite the opposite is often true. We should assume most app and website builders want users to return and want to appeal to users interests. Designing with users needs is central to the practice of UX design and research.

I think this question could be rephrased to. Why do twitter and Reddit use perceived anti-patterns instead of good UX? The answers being they can be effective tools to drive ad revenue, increase time spent with the service and increase user base to name a few.