As of right now, you can only disable the microphone completely, but I will look into taking care of this, as it is quite annoying!