We should really be verifying the fingerprint of the key itself, even if it is served over HTTPS.