That's not a fair assumption. Frameworks like Svelte, Solid, Vue etc have smaller bundle sizes and rendering speeds that approach the baseline vanilla-js cost.
I'm all for criticising Javascript, but moving everything to the server isn't a real solution either. Instead of slow React renders (50ms?), every interaction is a client-server round trip. The user pays the cost of the paradigm on each interaction instead of upfront with an initial JS payload. Etc.