Ahhhh. The key thing I was missing was `ssh -K`. This works, but isn't really quite what I was hoping for. It's effectively using the SK as a glorified flash drive to transfer the secret key. I was kinda hoping to avoid storing the private key on the local machine at all. There are upsides to this strategy however, especially given how simple it is and how it maintains compatibility with tools which require secret key material directly.
I may end up using a mix of this and a GPG integration once I learn more about how that works with SSH. I still need to dig into PIV/PAM/Keychain/FileVault/etc...
Thanks for the top though :)