The argument is that without your phone, you likely have no recourse to stop the attack. Since your passkey on the phone is what controls your access, now.

Yes, that's also bad. They're both bad. Passkeys are worse.