but for the past couple years I've just generated and forgotten 90% of my passwords. the final 10% I keep in a password manager. But if the service isn't really that important I just use the 'forgot my password' to change and generate a new password every time I need to login
In the end for a lot of services controlling your email is defacto controlling the login.
I NEVER log into my mail from my laptop/desktop. I access my email via my phone's mail app.
So
1. try logging on via my laptop's browser
2. service sends a magic link to my email
3. click the link on my phone
4. now I'm logged in on my phone! not what I wanted!
Even though i understand your consideration of separating regular access and reset onto different devices, im am still more sceptical about smartphone security than anything else. What happens when someone gets access to your phone? They could redirect and use the magic mails too.