Free movement is at the heart of the EU project. You start restricting that and the EU will tear up the agreements. We saw this already some 10 years ago when a similar vote passed, and EU stopped a lot of collaborations with the Swiss.

It's clear to everyone involved that the EU deeply regrets how they did the Swiss agreements. Switzerland basically ended up in a parallel carbon copy of the EFTA for all intents and purposes. It was clearly stated during the Brexit charade that the bureaucratic fatigue of having so many cherry picked agreements with Switzerland made the likelihood of doing the same with the UK basically zero.

The EU has always been clear that the single market comes alongside the four freedoms. If Switzerland approves this referendum that's a very high chance they will have to exit the single market too, which will hurt Switzerland immensely.

This is very ironic to me, because right now we live in a world where the irrelevance of small nations is getting clearer and clearer by the day. Political norms and international law are being trampled daily by the larger powers, and Switzerland was recently at the very end of it when Trump basically bullied them by imposing on them tariffs that were vastly higher than the EU and then ghosting them because he couldn't give a shit about them