The same thing happens with age-plugin-yubikey. You can only generate identities on the device (so you never have access to the private key), and the YubiKey needs to be plugged in and available in order to decrypt, as the core asymmetric decryption operation occurs on the YubiKey. (You don't need the YubiKey to _encrypt_, just the recipient string).

The way this is set up in the plugin protocol, the age client provides the age header to the plugin, along with any identities the user has asked to decrypt with, and the plugin returns any file keys it is able to decrypt. The process of taking the per-recipient encrypted blobs in the header and extracting file keys, is entirely up to the plugin. This falls nicely out of the way the age format's "one joint" is designed: the file is encrypted with one symmetric key, and that key is encrypted separately to each recipient via whatever mechanism their type requires.

If you squint through the layers upon layers of packet complexity, this is the same way GPG encryption operates [0]: the data is symmetrically encrypted with a random session key, and that session key is asymmetrically encrypted to each recipient. Only the asymmetric decryption part happens on-device for GPG; the symmetric part happens on the computer.

[0] https://datatracker.ietf.org/doc/html/rfc4880#section-2.1