Another issue is that most genetic modification work is done by massive companies, which then patent their results. If they develop cheaper-to-grow crops, farmers may essentially have no choice but to license their IP to stay competitive. That gives those big companies a lot of clout over farmers and probably advantages giant farming operations, who can negotiate to buy GMO seeds cheaper than small farmers can.
Some people also worry that there will be "bugs" in the genetic code introduced into plants that will have unforeseen consequences. On some level, plant genetics always been a software problem, but now we're potentially deploying updates in the (literal) field much faster. If new corn seeds distributed en masse accidentally produce toxins, or are unusually conducive to harmful bacteria, or just don't grow properly in some areas, what does it mean for farmers and consumers?
I think it's a lot of the usual technology-upends-formerly-slow-moving-industry issues, but that's especially frightening for a lot of people when it comes to things we're putting in our bodies.