I also believe we underestimate how much that costs us too. Yes, it is good that fewer people are breaking their bodies on brutal physical jobs.
But there are many people in the world for whom their physical strength and dexterity are perhaps their most valuable asset. With the loss of those jobs came a loss of an avenue of meaning for them.
I could be wrong, but I have a hunch that the whole manosphere alt-right conservative thing going on in the US is driven in part by a generation of young men who don't find intellectual work to be appealing but don't see many alternatives out there either. It's telling how closely Gamergate is tangled up in that world and how many games are basically simulations of heavily physical prestigious roles.
A recent Onion headline really hits the nail on the head: "'This Here Is Probably Our Bestselling Love Seat,' Says Man Who Would Have Been Powerful, Revered Warrior 4,000 Years Ago"