I don't know what this abstract "idea of copyleft" is, but the underlying principle behind the GPL could not be achieved without copyright. The GPL doesn't merely embody some notion of "right to repair" or "right to tinker." Its principle is to put the user on the same level as the original developer. It therefore requires not only source code, but source code in the "preferred form of the work for making modifications to it," plus everything required to build that source code:
> The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.
A world where you could freely modify binaries and distribute the results would not provide the same freedoms that are essential to the GPL. Copyright is absolutely necessary to accomplish that goal.
The GPL also embodies the fundamental moral theory of copyright: "I created this thing, so I get to decide the terms on which you use this thing." The GPL doesn't just embody the idea that users should be free to do what they could in a state of nature. It is fundamentally predicated on the idea of enforced reciprocity.