Sorry, while it requires unrestricted access to all my contacts, I'm not using it.
So yes I would prefer to have the option of adding contacts separately.
We could use something like ENUM mapping [1] on the device and there would be one single set of data that the user is in control. Even better, it would mean that I can choose the granularity of access to different communication channels, so if someone has my phone number they wouldn't be able to know my other messaging handles unless I make it public.