From section X.B:
"GMOs have the propensity to spread uncontrollably, and thus their risks cannot be localized. The crossbreeding of wild-type plants with genetically modified ones prevents their disentangling, leading to irreversible system-wide effects with unknown downsides."
Now, the very reason crop plants (GMO or not) cannot spread uncontrollably is tightly coupled to why they are crop plants. They are not poisonous, they are easy to eat, and they produce a good yield. A dream come true for every plant-eating animal. That's why they are crop plants. But they wouldn't stand a snowball's chance in hell in the wild nature, animals and insects would eat them quickly. Wild plants invest a lot of effort to not being eaten (unless they want to, for seed spreading purposes).
There are no cases of crop plants having invaded back to wild nature. Perhaps there is an odd canola plant that escaped a field and grows on a roadside or by an abandoned gas station somewhere, but these are also situations where humans first cleared the wild plants and bared the soil before the canola was able to set roots.
Crop plants don't spread outside of the farmers' fields, and GMO breeding is not trying to change that. (If they wanted to breed competitive wild plants, they'd use wild plants as the starting stock.)
I still think that a fear of "irreversible system-wide effects with unknown downsides" will persist, despite the good sense of your argument.
Also, as a technologist, I know that we move from solving easier problems to more complex ones. Who is to say that GMO tech isn't just beginning in farmers' fields, and that flush with success in that space, all those engineers, technology and know-how will happily move on to the Next Big Problem. This might be in wild-plant space.
Those are the true systemic effects wee should be looking into - there is no way back to 1700s farming now. (Organic farming is not there either.)