You don't own somebody's phone number because you have it in your contact DB. You are willingly giving them the phone numbers of everyone in your phone book... your
contacts are
not giving their consent to having their data sent wherever.
DRM would, hypothetically, solve the problem of giving your phone number to somebody but not allowing them to share it further without your consent. Except, of course, it doesn't.
By the way, because after years on the net I can see this coming from miles away, let me draw a distinction in advance between "I personally don't care about my number being shared without my knowledge out of other people's contact book to arbitrary third parties", and "I don't think anybody else should ever care about having their number shared out of somebody else's contact list with arbitrary other third parties." They aren't the same thing, and if you want to argue the latter point, that can be done without trying to argue that there isn't any consent issue at all, which is simply false. Whether you consider it right or wrong, some things are happening that some people don't want to occur.