I grew up in a farming family, and I run a hobby sheep farm.
Supplementing your food supply with a family garden isn't that difficult. Completely supplying yourself and your family with enough calories to live via farming is enormously difficult, both in terms of physical labor and in terms of skillset. It also takes at least 45 days, from a standing start, to get anything edible, and that's if you are willing to suvive completely on turnips (which have the shortest yield period of any domesticated vegetable crop). Corn, beans, and potatoes, which are going to be the crops that provide the bulk of your calories in a survival situation, require roughly 100 days to yield, and they only grow during the right season.
Furthermore, without either industrial fertilizer or an established farming system that includes animals, you will only grow those crops once before your land begins to be depleted and your yields in subsequent years will drop until they are no longer survivable in approximately year three. You are growing the animals as much or more for manure as for meat.
And none of that includes the vagaries of the weather and various plant diseases that can decimate your crop.
Basically, if you don't have access to fresh water, you're doomed immediately. If you don't have at least a 12 month supply of food in your basement, you're doomed in two to three months. If you haven't already implemented a closed nitrogen system using composting and animal agriculture, but happen to have land, seeds and a hoe, you're doomed in two to three years. If you have done all of those things you can probably survive for a decade ot three, if you can outlast the looters.
For reference, the pioneers who cleared the forests in the midwest would go in on year one and girdle all of the trees. Then they'd go back home. In year two, they'd plant turnips and pumpkins, which would grow among the dead trees, and they'd survive on those for the first couple of years as the stumps rotted. It took several years to establish a sustainable farm. The pioneers who went eg to Utah, where there were no forests, would plant immediately, but the first crop was just seed potatoes, which they would harvest and store - they couldn't carry enough to eat, and were essentially bootstrapping a potato crop from what they could carry. They carried enough food to survive through a complete growing season, augmented by hunting and foraging. It was a hard life, they were living hand to mouth, and they already knew what they were doing.