Honestly that's a sign you shouldn't stay around those people. If you're financially dependent on it and can't leave, okay, exception granted, but that kind of behavior isn't ok.
Sounds like terrible people. I’ve worked with plenty of people who didn’t start with English and if you give them time they usually excel