The 'limits' imposed by the GPL just limit people from reducing the freedoms provided to the code that is covered... So it limits your ability to use GPL code to reduce the freedoms of others.
In GPL-speak freedom means code transparency while you're defining freedom to mean something else (the ability to do whatever you want with code... including reducing GPL'd code transparency).
It sounds convoluted to try and explain this but to me it seems pretty simple and self-explanatory.