>I also really hope that hardware tokens like a yubikey are not required for every site or app. I'd like to be able to keep private keys on my phone or laptop for some things

Web Authentication supports this with what's called "platform authenticators" - some kind of TPM/secure enclave etc. built into the computer (most likely a laptop/phone), possibly integrated with a fingerprint scanner. The expectation is that sites will let you register more than one credential (like many (most?) do for U2F), so you can have a keyring device for initial logins on new computers (or for logging in on a friend's computer) and then use a platform credential on each for most daily use. Intel's built-in U2F thing is something to this effect, and might be compatible.

It's also theoretically possible that a phone could expose its platform authenticator to other computers via Bluetooth/NFC/USB, but that's still hypothetical at this point.