Here's a question: who owns your drivers license? Here's a hint: it isn't you. Can you "own" you mailing address? Copyright and trademark it, make everyone ask permission from you before they write it down? What about your salary? Should your employer have to ask every time they use your salary number in some way, say in aggregate statistics or reporting?
What about information about how you interact with your credit card company? Who owns that, you, or the credit card company? Do the two of you have some kind of joint ownership?
We've made some of these decisions about health data and it has far-reaching consequences, some of them undesirable. It's also been very difficult to enforce. Do you want to extend that kind of regime to every piece of information about a person? Society might grind to a halt, we would be inundated with virtual and physical pop-ups asking "your landlord wants access to your phone number to place a call to you, will you allow it?" And what process would mediate this access control anyway, and how would we trust it?