SFTP, like SSH, is "trust on first use". If you don't have some out-of-band mechanism in place to verify the server fingerprint, you're going to have a bad time.
While the CA system isn't perfect, rolling out your own CA to clients is easily automated and verifying certificates from that point on happens automatically.
I'm not talking about theory here. I'm talking about what OpenSSH can do.
> Is there any SSH server that accepts an X509 certificate and a client that accepts a database of CA root certs and checks the server cert?
Actually, it may not be X.509 per se. I'm not sure if OpenSSH uses precisely X.509 PKI or if it rolls out its own CA implementation, because I don't use this function. I've read somewhere it's the latter case.