My response is that everyone should carry. If the Klan goes nuts, the Klan gets shot. Eventually the system will react an equilibrium.
I agree with your freedom of association. My only note would be that removing people from the means of financial transactions is the same as limiting speech.