Copyright didn't come out of nothing, it largely evolved out of censorship regimes. Having copyright being about protecting authors economic rights instead of being about censorship or printer's ecconomic interests, was a huge step forward.

So mostly it is restricting natural property rights.