With the exception of Signal, I never give apps access to my contacts. If you think that's paranoid, remember how apps like LinkedIn used this kind of information in the past[1]. Thankfully on iOS apps almost always work without contact access (is this mandated by App Store rules?).
It's not clear from the article if the app just asks for access to contacts or requires it, but if you can't make your Voip app work without accessing my contact list, I don't want to use it.
[1] https://www.theverge.com/2013/9/21/4756212/linkedin-accused-...