The problem with "not caring who you're dealing with", is that it raises the possibility that who you're actually dealing with is a man in the middle, who is snooping on your traffic. If you get warned that a certificate is self-signed or signed by an untrusted authority, then you either have to check the key fingerprint or take a chance. Not that I'm happy with firefox taking this choice away from you, mind.