Javascript is one of the major reasons, but also trackers, huge graphics and loads of videos. Old pages still load fast so it's not the browser's fault. Say what you want about
https://www.lingscars.com/, but it loads instantly, is full of all animations and doesn't lag down the browser.
The problem is modern design, basically. We've invented server-rendered pages, then started rendering everything client side with frameworks like React, then we've started pre-rendering React on the server and completing content in the client, and now the pendulum seems to be swinging the other way around again to technologies like Flutter that just render an application in a canvas shudder.
People want interactivity, and developers want that interactivity to be consistent across their website. To do so, they need to recreate and emulate everything the browser does. In the case of React and other JS frameworks, that even includes constructing fake DOMs. Back buttons get overridden, links get turned into buttons that do custom routing, you name it and there's a layer of Javascript you can download to avoid having to do the hard work.
This is partially because of how demanding people have become. They expect any proper web page to be like Facebook. People, especially customers, want features, and they want them fast. Big companies that have their own developers are pushing their users towards their apps, sometimes intentionally sabotaging their website (looking at you, Reddit) to force people to download their invasive, native code.
Websites have become applications, and applications are inherently taxing on most systems. Google tried to combat this in their own, misguided way with AMP. If it wasn't for their stupid caching architecture, I'd be a fan of AMP because of the speed it can provide compared to "modern" web pages.
I don't think any of this will change unless we convince web developers to stop relying on all of these "modern" technologies and just write proper web pages. From the reactions here on HN, there are two groups of people in this debate: the people like us, who lament how slow the web has become, and the people who will never give up their fancy frameworks because of the productivity it allows them, and will never give in to the web Luddites who want to take away their fancy cross-platform tools that run just fine on their $2000 M1 Macs.
There are also a lot of web developers out there that are just shit, but we can't help those.