"If you always have your phone on you..."
How is this clearly not a deal-breaking problem? Tons of people don't have their phone on them 24/7.
Lets just say I lost my phone. Now I want to log into iTunes to use the find my phone feature. But I don't have my phone to use to auth.