If the server is compromised then the attacker can probably just grab a copy of the private key when it's decrypted for use.