But in this case the house itself is moving.

If Mercury and Venus had people, which of those people is our closest neighbour?