Before bilateral agreements and the freedom of movement, not Schengen which was ratified much later and is completely irrelevant here, you needed a work permit, not a visa (lowercase), which anyway at CERN is the equivalent of a diplomatic permit given to all international and tax-exempt NGOs in Geneva/Switzerland. And of course you lose your CDL permit quickly after your contract expires.
Getting a B permit before FoM would specifically not have been as hard for you as for someone from another continent.