Your comment made me realize that there are people who curate their browser history (other than just removing problematic stuff). Not criticizing, just never considered that was something anyone cared about.
If you curate your browser history, this is an easy fix. This is more of a problem for people like me who occasionally check their browser history because they want to revisit a site they were on recently.
It made the back button useless for me.
I didn't experience that using Chrome. What browser are you using?