Demand for interactivity. A real demand - that’s why desktop apps are now web apps.
It can be over applied, sure, but there are great reasons not to serve html templates. The trend is not some massive incompetence.
In my experience, the percentage of large react sites where someone sat down and seriously considered using old fashioned server side rendering, but decided on React because the site actually required a huge amount of interactivity is close to zero.
Every React site I’ve ever been involved with in in the last say 7 years was using React because “that’s the way it’s done”, and anything else was just a post hoc justification.
So many websites now pretend they're in the former category, but would work just fine if they implemented a less-interactive form. That's how it was done in the old days! "No, you can't have that. Here's the best we can do." "Oh, that's fine."
But I expect more of what's driving complicated backend/frontend split design these days is shipping the org chart (in companies large enough to have frontend and backend teams) and technology-by-consultant (where the right technology is whatever you can get a consultant in).
I actually somewhat knew React before this project but besides not wanting to overcomplicate things I was hesitant because I didn't like pure client-side rendering. Then I learned NextJS makes it easy to mix/match client-side rendering and server-side rendering so I just switched to that.
So now, even for a simple website, I'll probably start with React because I don't want to dig myself into a needless hole.
I wrote about the experience here:
https://www.billprin.com/articles/why-i-ditched-django-for-n...
And yes that post upset some people who thought I should use htmx, but React is actually pretty easy and simple. Now most of my websites are React because I know it so it's easy and simple to use what you're used to.
Also, "I did it in React because that's the way it's done" isn't the worst reason because you benefit from that popularity, e.g. some library you need will have clear examples of integrating with React.
I think the frontend and JS community have certainly gone off some crazy rails at points, but I also think there's a popular sentiment on HN exaggerating how off-the-rails things have gone, usually expressed by people who don't actually develop many modern-looking websites. Most of the trends have gotten popular for somewhat rational reasons, and even that stuff that has gotten a little crazy like an explosion in dependencies, is really just about tradeoffs e.g. lots of dependencies cause lots of problems but also enable code re-use so it has pros and cons.
But I've been doing this for a long time, and that's what everyone says. I've heard this argument at least 100 times when someone is trying to defend their decision to build something overcomplicated. The number of times they ended up actually being justified is so small that I'm very suspicious of this argument.
If you're not expecting to need fancy client side stuff, does next.js give you as good a server side development experience as Django/Rails/Laravel? At that point, it would seem worth it "just in case", as you suggest, but in the past when I looked at it, the SSR stuff still felt a little cobbled together and experimental. I would love to hear from someone with firsthand experience switching, though!
I think Next has a great DevEx, but I'm just using it for some projects as a solo dev without a ton of traffic so perhaps there's some rough corners I haven't experienced yet.
I do think they are going down some risky paths where they do things like rewrite your code files depending on whether you're on server or client, and in general with "framework-defined infrastructure" where if you write certain functions the framework "knows" that it's supposed to be on the server or in a background job or whatever. But so far I've gotten the benefits without the drawbacks, though I might be jynxing myself.