Which is, among other reasons, why you encrypt your key with a passphrase.
Why not just use the passphrase in the first place? Or do you think an offline attack on a key passphrase is going to be more difficult than an online attack to a server with rate limiting configured?