Another issue may be the social aspect of manual labor. I worked on a plastic assembly factory, with a more natural mix of male and female coworkers than any tech job I've had.
The pay and shifts sucked, but at least it felt...natural. Work your shift, grab a brew, and go out with a few of the mates after the gig. Good times.
I had a friend who was a almost-stellar IT career, depending on how you define it.. Anyway, he was doing well by most measures. In his late 20's he quit to become a ski instructor and train to compete in cross country.
He had a great time, got to do some physical work -- however, the main thing was that it made no difference to his career.
I think if you said to a lot of people - take 2 years out, climb mountains, be a landscape gardener, whatever - they would think this would be career suicide. The thing is that's rarely the case, and from the evidence I've seen it's actually the opposite (i.e. people who come back are well regarded).
Pretty much anyone I talk to late in their career (60+) usually say they wished they'd taken 1, 2, 3+ years out to do other things.