One key difference is power consumption, and to a lesser extent, data consumption. It is not practical for your mobile phone to continually spy on everything you say, due to battery issues; however, it can listen for a keyword to wake up to, and then do the more high-powered voice recognition. So the scope is limited, or you will just have a dead phone in your pocket (which, of course, will also not be listening to you).

With something like Echo / Home, the always-on power source makes it technologically feasible for it to always be listening to you as well, without needing to wake up to a keyword. Now, they may as a feature only start processing based on a keyword, but you don't have the "comfort" of knowing that it is technological limitation rather than a software decision that may or may not be in your control.