If you traveled back in time, the earth would had
moved on, not to mention the solar system and galaxy. Wouldn't you have to travel in time and space?
Aren't you also increasing the mass of the Universe? You could add enough mass to halt the expansion.
I think it's just easier to assume time travel is impossible.