The margins in most farming are razor thin. Our neighbors who grow conventionally spend tons on inputs like fertilizer, for a shockingly small amount of money [in return per acre]. A year of extreme weather – more and more common – throws the whole thing out the window.
People say they want organic, but then they balk at the price without realizing how much labor goes into it when you're not just spraying to control weeds (or, more shockingly, to stop growth on your food crop at just the right time).
All of that said, a lot of the negativity directed toward anyone who has the dream of growing their own food is often coming from a conventional mindset. There are alternative approaches. A few great books are "Permaculture" by Mollison, "One Straw Revolution" by Fukuoka, "Restoration Agriculture" by Shephard.
As I mentioned elsewhere in the thread, it doesn't have to be all-or-nothing – you can start with some perennial herbs on your balcony. Larger scale, I'm an advocate for things like agroforestry practices, becoming more and more interested in agrivoltaics. You can go a very long way with a few dwarf fruit and nut trees, an understory of berries, and a few raised beds managed with no-dig methods.
If nothing else, you learn first-hand the challenges (and joys) of growing food, become more connected to the world that sustains us, and maybe gain a better appreciation for the people who work really hard for very thin margins to keep us all fed.