Wifi spectrum usage is also managed to a large extent by spatial separation. There are still fairly strict power limits on 2.4GHz devices which ensure that they have a relatively short range, so while your neighbor might interfere with your wifi, neither of you are going to have any effect at all on someone even half a mile away.