> because a hypothetical light particle emitted at A couldn't reach B in time

Fascinating insights - it can be seen as causality, or at scale that there is a multi-dimensional reality where some timelines are inaccessible/impossible to each other (yet exist in their own right).