Your presumption is wrong. I gave examples of free movements of products, services and capital. If you are ready to accept these freedoms, you should also accept the other items packaged in this very same deal: the free movement of people.
The Swiss also like to cherry pick the good stuff from the Europe, just like everybody. Not taking responsibility of the economic externalities, as the euphemism goes, is not sustainable. Economically speaking of the West, the short-term advantages of this short-sightedness are over.