IIRC training starts with the initial state and the end state, and the end state is always oriented the same way. It would be interesting to see what would happen if the end state was rotated randomly though I suspect it wouldn't work so well.

yeah normally NCAs have a sense of up and left. There are some isotropic variants that make the perception fully rotation-invariant.