Right, that's why I mentioned that it's less the case today, but that freedom of movement is always at risk of being cancelled much more easily than a country leaving the EU. There has already been a referendum that passed that would have done so. Also, non-Swiss Europeans can be expelled for certain criminal convictions and (as you mentioned) need to be able to support themselves financially which aren't generally the case between EU states except in extreme circumstances.