Unless you write down that password or use it elsewhere, that can be considered secret. Compare that to the difficulty of sampling someone's voice by simply eavesdropping.