They are both bad in different ways. The first one is meant to listen all the time.
Besides, many phones listen all the time too.
The only real difference is that the Echo has better microphones than your phone and is powered on all the time so you don't have to worry about battery life. On the other hand you probably say more sensitive things to your phone.