Unfortunately I will have to move a significant codebase out of Next.js to perhaps Remix as I run into all mentioned points.
This is not unfortunate. I've done this recently and I'm so glad I did. Remix is much, much, much simpler and pragmatic. And doesn't have all these artificial limitations Next.js has when you're running it on your own server. I can't recommend it enough.
If you want something “safe”, nowadays the only option is to use one of the big MVC frameworks (django, laravel, rails, etc...) with inertia.js, if you want to use React or Vue.