GPL is about ensuring that all users of the code, both developers and end users, receive the same freedoms that the authors of any derivative work had.
The freedoms awarded by the GPL are not intended for the authors of the derivative works but for everyone else. The restrictions in place only prevent the author of the derived work from restricting everyone else.
If you want the freedom to do whatever you want with your code, then write it all from scratch.