I don't think it's that deep: Obligatory manual labor destroys the body (and, often, the mind) and what time you have you spend exhausted. Being entirely sedentary remains a choice for us office workers—this is why people exercise and spend time outside.

Of course, I would like more flexibility in choosing how much I and where I do my sedentary labor, so I might devote time to, say, gardening. But it's easy to forget that humans have spent most of human history trying to escape subsistence farming.

I have worked subsistence farming for a small portion of my life, and I cannot tell you how hard it is, physically and psychologically. That was by choice, as part of essentially joining my wife's culture and family. If I were to do that for the remainder of my life it would destroy me.

Anyway, I'm going to go happily work from my desk 30 ft from my bedroom while drinking coffee likely farmed for about ~$0.30/hour while I make a few hundred times that.