> This quickly stops making sense when you're making an application, as you want that button to go "up" rather than back.

Did you try to use the history API [1] for this ? Not straightforward but with either replaceState or the onpopstate event maybe you could achieve with JS what you are looking for.

[1] https://developer.mozilla.org/en-US/docs/Web/API/History_API