Another 'reason' it works for odd-dimensional spheres is that the (2n - 1)-sphere can be identified with a certain subset of C^n (n-dimensional complex coordinate space) where your 'swap elements and multiply one by -1' idea is just multiplication by i, which, when you think of your vectors as being back in R^2n again, always produces something orthogonal to the original vector.
Even better, the (4n - 1)-sphere (so think of S^3, S^7, S^11, ...) can be thought of as a certain subset of H^n (same thing as before but with quaternions instead of complex numbers), where multiplication by i, j and k are available! And now in this case you have not only one nowhere-vanishing vector field on the sphere, but three everywhere pairwise orthogonal vector fields. This in particular shows that S^3 is 'parallelisable' — a property it shares with S^1 and means that there exists a continuous global choice of basis for each tangent space.