People always pull this argument out of their ass: "domesticated plants are genetically-modified by random chance, whereas in labs it's by design". The difference is, obviously, that there seems to be some kind of negative feedback mechanism in nature - organisms optimise towards the local maximum, but these small modifications are unlikely to result in something
globally optimised (well, at least so far in 3bn years, no organism has taken over the whole planet). With GMO, it's fairly easy to combine some optimal bacteria gene with an optimal plant gene and make a super-organism.
It's like saying, well we shouldn't worry about terrorists enriching uranium, after all due to random movement of particles a nuke could just assemble spontaneously. I mean, yeah, it's possible (in fact, natural reactors exist) but so so unlikely; on the other hand, humans managed to do in in a decade of concentrated effort... several times.
I'm not even saying that the above scenario is likely, or that legislation can prevent bad actors from creating super-organisms. But the point is, there's thousands of similar scenarios - we simply have no idea what even single genes/proteins do, let alone combinations of genes from species that have had billions of years of distinct evolution.
But it's the tail/catastrophic risk that worries me. It's like with climate change - almost nobody would mind 1 or 2 degree celsius increase, there are plenty of benefits actually - plants works better with increased CO2 in atmosphere, and maybe we could even grow food in Siberia... but it's the runaway climate change that's worrisome - i.e. it's impossible to guarantee that it will be only 2 degree increase. And if we can't even model physics/climate, how can we model biology?