No, I won't leave Facebook entirely over this, but I did just uninstall their application on my Android device. I simply don't trust them to have access to my contact list.
Frankly, I didn't trust them much before: I'm sure they were mining it, which is not something I would explicitly consent to. However, now I no longer trust them to not break it, so I've removed their access to it entirely.