"People focus too much on the 'work' aspect of it, it should be looked at as more of a zen practice. Growing and preparing food is very zen and is a very karmic and devotional practice!"
In this way, one could conclude that the work you put into it actually does cost nothing and in fact benefits you through it's karmic ROI.
When I'm at work I find myself looking forward to getting home so I can spend my 20 minutes doting over my plants. It's also really rewarding seeing how much everything has grown by since yesterday (sometimes it's by surprising amounts).
At the sort of scale I'm doing it... it doesn't feel like work at all. But I can imagine that changing if I was depending on it for survival. If I scale it up, I very much intend to rely on supermarkets as a backup (especially as I'm still learning). Eliminating the stress (and work aspect).
I would love to have a large multi-level greenhouse to work with. I suspect that the average backyard could supply enough food for three or four families with about 1hr per week invested in time.
When you set up a hydroponic system like that, the cost side of the argument in the OP changes dramatically though. Then it becomes a real 'hobby' in the sense that it costs more than you can possibly get out of it (unless you grow marihuana in it - ironically, the most information I could find on hydroponics was in books for marihuana growers, and the shop I got my supplies from sells their stuff mostly to that audience and demos their gear by growing peppers in them. The first time I came there the guy was quite excited to speak to someone who didn't intend to grow marihuana).
I get pretty upset by this sort of 'reasoning', since it dismisses so much suffering by people who actually have to live a subsistence farming life, or barely above that. It's usually made by rich yuppy guys who have never known any hardship but who feel the need to proclaim the moral superiority of 'back to nature' or 'local food' or whatever. Fuck those people, with their know-it-all attitude - millions of people who actually live the life say you're full of shit, go do it yourself before you go all holier-than-thou on the rest of us.
(related: a guy at a conference on sustainable energy I was at a couple of months ago was preaching the sustainability gospel and mentioned the Uros in Lake Titicaca in Peru, who live on floating islands made of reed. He held these people as some form of example because of their lifestyle which is so entwined with nature, much better than us Evil Westerners. Well I have actually been on the Uros, and the people 'living' there are, apart from some old people who don't want to change what they've known for their whole lives, actors for the tourists, and live in Puno (big-ish city on the shore of the lake) and in the mornings put on their reed skirts and take the first boat to the islands, and then meet the tourists who take the next boat. I know because I was in the first tourist boat of the day and some of the guys missed 'their' boat and had to come with us to the island. Who wants to live on a floating reed island, catching and eating fish every day when you can have 5-day working weeks, a TV and a car and send your kids to school and soccer practice? I found it incredibly insulting that this tree hugging idiot somehow dismissed these people's drive to move away from their primitive lifestyle and wanted to build a live of modern luxury for themselves. All while he himself didn't fail to mention how his own house 'was superbly insulated, and very comfortable' - yes you fucker, you live a comfortable life subsidized by our governments (he was a professor) and then you go preach how a 'primitive, natural lifestyle' is so morally superior.)