Unlike the other two, "frat" has no alternative denotation and is a common alternative to fraternity. By your logic we wouldn't be using "app" or "admin" or "limo" or "memo."
i notice a lot of fraternity members make the "you wouldn't call your country a ...", guess it's just the common chant (amongst others), though i don't understand why they consider 'frat' a derogatory statement