Don't you think the owner of this deserted island with enough resources to live on might at some point object to you being there and force you off his land?

There is no living outside of society, outside of private property and the rule of law. There is no real choice here.