The physicality was a nice change from sitting in a chair, and the work brought benefits of its own: improved skill at problem-solving and efficiency (the former more in construction, the latter more in cooking, but with elements present in both), improved people skills, a better work ethic and appreciation of safety (and by extension, responsibility to your colleagues) and better work-life balance - preparing or cleaning & putting away the tools of one's trade every day became a useful ritual that helped to define the border between personal and work time, something that might be difficult when you can do system administration via your phone.
Of course I made much less money, not least because I lacked commercial experience or special training. But I learned a great deal in the process, and of course the task-specific skills can save one a good deal of money - for many home-improvement jobs I don't need to hire a contractor, and even if I do want or need to, it's a lot easier to estimate the scope and likely cost of the work. To this end, I'd say choose a job that requires skills and will challenge you, even though you might find it more difficult to get your foot in the door (especially in a poor economy, where employers don't want to hire over-qualified people who may flee at the first opportunity). So while being a waiter might be challenging and might improve your ability to handle customers, you won't accumulate a whole lot of useful skills in most waiting jobs. But many other areas allow you to develop meta-skills that will be useful in any job as described above, and if you like the industry you might well see market opportunities for computers, should you decide to return to hacking at a later date.
Also, when you do you may well earn more money as long as your computer skills haven't atrophied. Many employers value diversity of experience if it brings a deeper perspective and if it was undertaken from self-development rather than out of necessity. I found I enjoyed other sorts of work besides the computer-based kind, and when I was recruited a couple of years later to a computer business again, I had to think hard about accepting. Unexpectedly, my would-be employer interpreted my hesitancy as a negotiating strategy and upped his offer by about 10% on the spot. Also, I didn't go back to the same old grind I had felt tired of before, but in a direction that employed some of my newly-acquired skills (more customer-facing and process-oriented) and continued to look for new areas of development - in my case that was learning about multimedia hardware which eventually took me into the entertainment field.
I never quit suddenly, going days without turning on the computer. But I do wonder sometimes if this is the best use of my time. Is it really worth typing this to you? Why the hell am I not at least trying to have sex with two world misses?
In the end, I've decided that I quite like my life the way it is. I love my fiancee and I plan to have four kids with her.
I guess it's matter of finding what you want. Now, knowing what you do NOT want is useful too. Just act on it. Start with a few things that you've ever wanted to do, like kung-fu in my case, and develop into a hobby, a passion, and maybe someday you'll earn money to do it.