I agree to harvest your crops in exchange for 50% of the yield.
I agree to sing if you give me supper.
I agree to sing if you give me supper and not record me.
I agree to sing if you give me supper and subscribe to all the rules of copyright law.
I agree to sing if you give me supper and subscribe to all the rules of copyright law and all people in your jurisdiction also subscribe to copyright law. (And then some folks who wanted a singer got together and passed a law that satisfied the singer's requirements).
All these seem reasonable to extensions of the previous and it seems like we've ended up at the latter.
You might say 'well I didn't agree to that contract', but there are plenty of things in the world that we are bound by that individuals that we didn't explicitly agree to. I didn't explicitly agree to not dump motor oil in the ocean, or hunt deer on a certain day or that 50 decibels as the bound for noise during the night, or that children of age 12 can't vote, or that I have to wear pants in public.
You might argue that the last step is invalid, and instead the world should be a place where individuals can pick and choose which laws they are interested in subscribing too. I don't really want to put much effort into that path because my intuition is that's equivalent to anarchy, but maybe there's an alternate argument here.
So it seems to have a world w/o copyright law, you have to throw out contract law, or convince everyone to stop somewhere on that chain, perhaps not to make the contracts that mimic copyright law, or make a law making such contracts invalid. The later two options seem fairly artificial, impinging on folks self determination. So we're left with having to eliminate contract law, which seems pretty limiting.