There is a workaround, but it isn't pretty. You can push/pull bugs and identities with normal, ssh-agent-less git commands: https://gist.github.com/parasyte/80ef925d4b01216f6bcb051356f...
Specifying identify files for specific hosts, using bastion hosts (ProxyJump), etc. For example, my laptop SSH config detects when I'm not on my university network or VPN and adds a ProxyJump through the department bastion host, but connects directly when I'm on the network. Configurations like that break when software makes too many assumptions about how ssh is configured.
also, don't know a good security researcher who uses agents (some even use ssh config to limit which keys are even tried per domain)
ssh config hacks are also a must if you have, say, two or more identities to github.com for example.
I use a Yubikey for some of my keys, and I've never used an "agent", don't know how, and don't want to start, unless someone can give me a good reason to
ssh-keygen -t ed25519-sk
that's all you need to use a Yubikey with SSH