This is not a vote for Switzerland to exit the EU...for obvious reasons. It is a vote to exit the Schengen.
As OP explains, freedom of movement can't be stopped in isolation from the rest of the bilaterals.
(btw funnily Schengen is just about the border control, we're talking about freedom of movement which is a different thing, e.g. UK wasn't in Schengen but the freedom of movement applied to UK as well before brexit, tho I guess people use Schengen interchangeably)
„Exiting the EU“ is a perfectly adequate way to summarize it to a world audience that doesn’t care about the details.
Incidentally, when brexit was being voted, the only person I knew who thought it was a good thing was Swiss. They are just fiercely independent.