I'm in your boat. Gene's are letters in a story. To say "That's a salmon gene! It doesn't belong in my tomato!" is puzzling, like "That letter belongs to another book! Keep it out of this book!"

Anyway, good ol' paleo food was whatever nature provided, most of what was happy to kill you. Humans tamed 1000's of plants over 50,000 years changing them beyond recognition so they wouldn't hurt us with alkalis and poisons. And so the tasty bits were bigger and easier to get at. All by genetics.

Sure we can do it faster now. But we're aware of what we're doing. Grandparents with crossbred tomato didn't have any idea if she was reintroducing a poison or raising the acidity to dangerous levels. They were blundering in the dark.

I see no problem with turning on the light, and moving confidently.