How do you accurately resimulate if you don't rollback everything to the previous state? E.g. a giant wall moved from point A to B but it was accurately predicted (no rollback). Don't you need to roll the wall back to accurately simulate everything around it?

EDIT: I misunderstood the previous comment, I think you are rolling back everything that changed, and not rolling back objects which were "static" in that timeframe.