I'm not sure stateful vs stateless is even the right way to frame it.
With no JavaScript at all, your application is still stateful. Even if an individual HTTP request is stateless, your app as a whole still has state. The state just lives in the database in that case, and each request you make either gives you a view of part or all of its current state or provides data to transition the application from one valid state to another valid state.
Actually, thinking of an application as a series of transitions between valid states is why I enjoy using things like Redux. Used properly, it acts as a gatekeeper that makes it hard to move the application from a valid state to an invalid one. It adds a fair amount of boilerplate, though, which I know some people find frustrating.
Maintaining state on the front end and keeping it in sync with the back end is a pain. But I don't think it has to be. it's just an artifact of us being stuck in a certain paradigm of application development that we don't seem to be able to get out of.
It wouldn't be impossible to design a system that runs over a network, but doesn't require the front-line application developer to think about the network. It sounds like something Alan Kay might have worked on at some point. :)
I envision a Smalltalk-ish environment where, in my application running in a browser, I send a message to an object telling it to save itself. And when it receives this message, it transmits itself across the network, validates its data, and persists itself into a database. Maybe an object database like Gemstone/S. But the application developer wouldn't have to worry about all the details most of the time. Just tell the object to save itself, and it'll report back whether it was able to or not, so you can update the application accordingly.
I'm sure I'm overlooking a ton of potential pitfalls, of course. I think we can do a lot better than we're doing right now, but I'm not 100% certain what better looks like, or quite how to get there. Maybe I just need to watch a few Bret Victor talks for inspiration and then get to work trying to build something. :)