Most web pages are not hand crafted page by page, they're part of a CMS with a million other capabilities. Or, the team creating the website was under cost pressure and went for some performance-compromised "good enough" solution. Or, the website is old (true for almost all of them) and no longer under serious active maintenance.
It's not just incompetence leading to technical bloat.
The same is true for some of the "dark patterns" like auto-playing videos and pop-ups. We should stop with the narrative that this is due to publication truly hating users or being completely tone-deaf. It's purposefully done because it works. It serves marketing and monetary goals.
Yes, this comes at the expense of usability. You should see it as an act of desperation that is existential. It keeps the lights on. The terrible state of web usability is due to it not having any reliable business model other than the current status quo.