While your name and phone number are public, the list of people who have your phone number saved is not public. These companies can effectively map who you associate with only by looking at other people's phones. They can profile you by association, and that is very wrong.

At some point privacy is no longer a choice, not a real choice anyway. You get to chose between participating in society or keeping your privacy. It shouldn't have to be this way, but it is.

Oh, uh, I thought you were talking about your name.

Microsoft collects your friends' names so it can spell their names correctly when you use speech-to-text or related features. I thought that's what you were objecting to. What kind of data are you talking about?