Have you considered writing passage in Go to integrate age and age-plugin-yubikey as a single binary to make it more convenient to setup and use?
Integrating age plugins into this binary is not something I would do, though. Besides the fact that the plugins are separate projects with their own release schedules, they are also in different languages (plugin-yubikey is Rust, plugin-se is Swift, ...), and you would need to fork them anyway for communication. I guess you could bundle the binaries together in a package, and make sure the search path searches for the bundled binaries as a fallback.