This idea breaks down pretty quickly when you add more spaces. I use 9 spaces on my macbook pro and have created multi-touch swipe motions to move me in any virtual direction I want. The simpler solution is to just make the spaces an infinitely wrapping plane like playing asteroids.

For any one who thinks that 9 is too many spaces, back when I used lower resolution PC's I actually had a tool called JSpager that gave me 12-16 different 'spaces' and a miniview in the corner with views of the layouts in each space, and I could easily keep track of everything.