First of all, I agree.

Yet I think I have a stigma claiming illness or disability... don't know. So, next time I'm going to try this:

  I am a foreigner and have serious issues understanding 
  spoken English. Please avoid calling me and let us try to 
  resolve the issue over email.