If above average security clearance is disclosed immediately by someone seeing my badge then thats on the company, not the person with the badge.
They're not to be worn outside, and the employee policies state that.
You can argue all you want with your superiors about who it's on while you're being escorted from the building for gross misconduct.