IMO, a much more natural approach is to look at things from the perspective of light emitted. Specifically black body radiation. Red/Blue shifting correlates with changing temperatures. And temperature works as a natural proxy for time.

The other thing that confuses people is presenting things from an 'independent' perspective in the diagram.