This would be true if there were not repercussions for the wider Swedish model, but since there are repercussions of rolling over it is Sweden vs Tesla
This is specifically the case with the Sweden Democrats. A big motivator for them is a) bashing greens, and b) being a better alternative for the working class than the Social Democrats. Siding with a foreign billionaire over "salt-of-the-earth" workers is not a winning move.
Vocal or not, no one will not be able to affect this semi-voluntary system unless they control bascially the entire economy. This is unlikely to ever be the case, until maybe some supranational EU agreement changes the rules entirely.