That was my problem first too, but you can make it require the token on mobile too. I have it set up that way and I need to tap my Yubikey NEO to authenticate using NFC on Android. Works great.