Don't worry someone in your circle of friends already handed it over.
Exactly. It takes just one person who knows your phone number to upload their entire contact list and it's saved. The likelihood that Facebook doesn't have your phone number and name is 0% unless you don't have a phone number.
It's worse than that. If you gave them your phone number, you can ask to see it and have them to erase it. If one of your contacts gave them your number, it's _their_ data, you cannot get Facebook to confirm they have it and you can't ask them to remove it ...
'ye old 'sync your address-book' trick.
You can't be sure that Facebook doesn't have your number unless you have never given your phone number to anyone who uses Facebook and has your name and phone number. They could be friends, co-workers, relatives, delivery people, cab drivers and many other service providers. One of them shares their address book with Facebook and it knows. A few of them share their address books with Facebook and it's reasonably sure what your number is (names can be fuzzy matched to your profile).
This is why I hate that people carelessly let any third party app have access to their contact list.
I can keep my data out of Facebook,but "friends and family" usually aren't careful and get tricked into opening up all that info.
A couple of years ago the only social media I had was Instagram. One day I went to login and it prompted me to enter my phone number and there was no way around it. I decided to delete the app instead. Shortly afterwards my account disappeared. That was the end of that.