What prevents you from just using certificates not signed by a CA and verifying them based on the public key fingerprint?