Go to your Google account settings; add the languages you speak and don’t want auto translations for in your personal profile.

I agree that the auto dubbing is the worst feature. It may have been HN where I read the above tip to turn that off, it seems to have worked for me so far.