This is an issue for other messaging clients as well, having the "Sync contacts" enabled by default. So by the time you get in the settings to disable, your contacts might on a server already.
Thankfully now that permissions are incremental, this issue is solved.
Except some apps still target an old API in order to keep install time permissions .. like SnapChat.
This will soon no longer be possible since Google will force apps to target the last API level.
Still, people don't seem to realize what giving an app access to 'contacts' or 'SMS' means.
I am not sure how to solve that tbh :/