Oh but there's so much more energy in your food. There's the energy that went into producing fertilizer, energy to make pesticides, energy burned to run tractors, energy to process the harvest into what you get in the grocery store, potentially more energy involved in packaging. For plastic packaging, there's the oil used to make the carbon-rich polymers. Etc.
Just like cars, all of that energy comes from burning fossil fuels. The only way you'll find carbon neutral foods is if you walk out in the woods, pick it off a wild plant, and eat it right there.
Generally, the total energy that goes into producing and transporting something is called "embodied energy." It's not necessarily 1:1 related with carbon footprint since it can hypothetically come from renewable energy instead of fossil fuels, but it's a very related concept.