What we call 'the past' is a mental picture of where physical things used to be positioned and their condition. But everything in 'that now' has changed. Just as when we 'travel' in physical space, when we get 'back home' it has changed.
If I were to 'time travel' I'd be gone from now. But everyone else would go on living in now. So where was the 'instant of time' I travelled to stored so I could go there? And the infinite number of instants between it and now?
Fanciful idea, but no evidence ... none zero zip nada