It depends on the colors badges can be, apart from blue. If they can be yellow or green... that might be a problem because of tritanopia and tritanomaly. But I only know of blue and orange badges. So true... those orange badges might appear red to someone with tritanopia... but he's not going to mistake a blue badge with an orange-red one.
You are still right for the fully achromats out there. But a shading level difference between the orange and the blue badge could be enough. Otherwise, a simple indicative letter is enough (E for employees, I for interns and C for contractors would be more than enough).
Again... if there are more than two colors for those badges (green, yellow, red?) the problem of relying on color to convey information could be more problematic.