The same thing happened to me in China when I tried to speak Mandarin. Fortunately, however, there were many people who couldn't switch to English because they didn't speak it.