I used to freak right wing libertarians out by following this like of thought with them when I was politically active. I had one in tears during a break in a debate because I was there representing a Marxist party, yet I agreed with him about most issues. He could not understand why, and he saw it as destroying his credibility.
I pointed out that to me, as a marxist (some would argue the term libertarian socialist, or left communist would be more descriptive), his arguments for individual freedom was the right starting point for evaluating most policy, but the only logical conclusion is to limit government recognition of property rights too, as a large portion of the laws to protect private property takes away more rights from the public at large than what it offers to the public as owners:
I may gain exclusive rights to a small area, but I lose access and even limited rights to most of the world.
And In fact most countries do recognize that property rights must be limited to various extents to safeguard the rights of the population as a whole, through things like guarantees of access to land areas held in common (or owned by the government) that will not be sold, or legal protections for access to beaches or wilderness that often extends to granting legally protected public access to privately owned land.
The question is not whether or not property rights must be limited, but where to draw the balance. Yet somehow right wing libertarians tends to want to want to tighten property rights, while at the same time they argue for reducing government intervention in pretty much everything else. The inconsistency is glaring to me.