This leads to anti-minority laws like Minarett-Verbot (minarets can no longer be built due to this law) etc.
It was really mostly because people didn't like the architecture. Muslims in Switzerland are freer then in many other countries in Europe and for every 'evil populist' law that Switzerland made you will find that 10 populist proposals have been rejected (left and right wing).
Also, lets remember that Minarett are something people like, they are not required or in any way limit religious freedom of Muslims.
So yes, we had like 4 Minaret and they wanted to build 2 more and can't. Populism has truly made live hell.