It would be different if it worked like ssh, where key changes are warned about after the (optional) leap of faith.
You're correct that this won't protect against a MITM during this first visit and will cause confusion if an attacker pins his fraudulent public key. However, I would argue that pinning provides a large net security gain in practice. The same model of vulnerable initial visit but secure subsequent visits is used in HSTS to prevent ssl stripping attacks.