If CAs start to get backdoored, people can operate a la web of trust or other asymmetric protocol where the public key is posted on a public board and the server can verify its ownership of the private key without a third party other than the public notice. More work but should be doable.