Actually, it would still be here: Van Gogh artwork is unique, and this can be expanded virtually to anything, including the shirt you are wearing, since you're the only one to wear it. You might see where this leads to. Scarcity can always be created artificially.
Anyway, you wouldn't want people to take your shirts away from you, and copyright holders may feel the same towards their work.
The second issue is fairness: if someone has to pay a certain amount to access copyrighted work, why others would not have to, under the same circumstances? Fairness is the motivating principle supporting the existence of the Law.