This should be a wake-up call for the move to elliptic curves, as GPG and OpenSSH allegedly support them. Windows support is lacking for remoting into Linux hosts with elliptic curve based ciphers. I found that while PuTTY does not support this directly, evidently the Gpg4win agent can work with putty, and that might allow ECDSH/ECDSA. I haven't tested this, however. And another terminal client, Tera Term, purports to support elliptic curve algorithms.
Unfortunately for GnuPG, it looks like the only supported curves are those defined in NSA Suite B (the NIST recommended curves). That set is considered tainted by some community members. There's some discussion of GnuPG supporting Curve25519/Ed25519, of implementing it, but I can't tell if it's implemented or not.
Edit: I've played with Tera Term and can confirm that it works with Ed25519 along with OpenSSL 1.0.1f. I'm pleased with the result.