Because private certificate may leak and it will be difficult if not impossible to tell a fake from the real thing.
I believe at one point this happened with Covid passports in some countries.
The same argument could be made about the possibility of a CA’s key being leaked.
Also if the private cert is in the Secure Enclave it’s harder to leak.
But also your physical passport can be stolen