Using Node.js would hurt productivity but it's more powerful if needed. It may be needed because it has async/await and it has socket.io. It's also possible to use Typescript.
Next.js can be useful if you need everything (good SEO + highly interactive) but let's be honest how many websites need good SEO and websockets? LinkedIn maybe.
Next.js server performance isn’t great either - honestly, it’s pretty bad. Pages don’t feel that fast for users despite all the tricks and optimizations. In my opinion, metrics like LCP and others are much easier to optimize in older, more traditional frameworks. Unless you’re building a full-blown web application, a classic web page will almost always be faster and simpler to deliver without all the Next.js complexity.
That said, I'm not necessarily a big fan of mixed environments since IMO most web apps can be mostly client side and most websites can be mostly server driven. Either is simpler than the mingling tends to get. That's just my take though.
Nextjs with pages router was pretty good for me but app router is where it got complicated
If you're using Nextjs to get a high level of rich interactivity then your SEO issues aren't about CSR vs SSR anyway though. As you point it it's just bundle size and site speed, and URL structure that end up causing a negative impact. Nextjs doesn't fix that.