The RFC addresses security, but does not mention anything about privacy. I think the scheme ultimately boils down to trusting the server/instance.

It would be great if users don't have to share the actual number with the server, a hash or something like that but that would make it impossible to verify the number and verification is required to prevent spoofing.

Another way maybe is to have a trusted 3rd party (something like EFF, LetsEncrypt) that can be used by users to validate their numbers and applications can get the hashes from there.