So it's not about labels, but you want the ability to restrict the fields an app has access to rather than an all or nothing – full access to a contact or none at all?

I'm annoyed at everyone who shares my name, phone number and any other details with Meta. I never consented to it. The behavior of their app slurping up your contacts database is despicable.

This doesn't answer your question, but in case it helps for others out there: it's possible to use WhatsApp with no access whatsoever to your contacts and I used it that way for years. Connecting with people is slightly jankier but it still works.