All this assumes that time is a single thing (sort of gigantic X-axis with "t") that works for everyone in universe equally.
What if we have multiple (billions) of realities with their own t-axis and with us being only one of these with our own timeline?