You should be using POP or IMAP only on your phone so that you can revoke permission after it is stolen (when you log in to your account from a desktop and use a backup key from the printout). If you are logged in to your main Google account on your phone you are asking for trouble. While I will be sympathetic after it gets stolen and someone ruins your life, I won't be surprised.