You can maybe, trust the user to handle it's own certificate in their own devices?
Though I admit requiring attestation is probably a good default.
But yeah, the user could have a choice to this extent.
That's what smartcards like Yubikey do, my government certificate is on it and it can't be exported. They could attest that but beyond that, the operating system of the host device is none of their business.
There's no proper way of doing so on Android.
Some countries, like Estonia, are providing their own SIMs to solve this problem. That indeed works. Unfortunately phones are being made that are eSIM-only and certifying eSIMs to the same EAL level is near-impossible.