No, I started out with violin, then moved to piano, did nothing for 5 years, then played saxophone for 17 years and then stopped for a couple of decades.

The big difference between violin and say piano is that with a piano if you hit a key with a certain velocity it will make a certain sound. Over and over again. On the violin it isn't nearly that simple and just forming the tone will take a long time (in a way you never stop getting better at that).