Skilled work is not the antonym of hard work. You can work hard doing skilled work.

Knowledge work does look different from physical work, but you can work hard at both; you need to work hard at either to be successful.

As for why hard physical work is so valued - only a century or two past, it was the only practical method to prevent the starvation for yourself and your family. Knowledge work being a viable means of survival (for non-nobles) is pretty new, all things considered.

Even my own parents never really understood how knowledge work could be as valuable as getting out and working with your hands; and they were born in the early/mid 1900's.