Frameworks solve these problems efficiently. Sure, some frameworks can be overly complex (I'm personally not a fan of Next.js), but that's a problem with the specific framework, not the idea of using a framework at all. But many frameworks make things much simpler and let you avoid reinventing the wheel. You could write your own scripts (in practice a mini-framework) but eventually you'll hit limitations, especially as your project grows or you start working with others. At that point, you'll probably end up switching to an established framework anyway.
If you're already using something like PHP with server-side rendering and templates, that's fine too—you've just chosen a different kind of framework (it is still some kind of framework. Just not a client-side one). I just don't buy into the idea that avoiding all build steps and frameworks is somehow more "pure." It feels a bit like a hipster take: "I'm going to write everything from scratch and avoid all tools just to show that I can."