Life was hard when we had to forage for food.
Life was hard when infant mortality was massive worldwide.
Life was hard when we didn't have a cure for polio.
Life was hard when the economically disadvantaged generally just quietly died out because no programs existed to help them over a bad chunk of time.
And life is now hard when you get cancer, lose your job, or are forced to move because the economics of your town have changed and you can't afford to live there anymore. Worth noting is that sometimes, these three are inter-related. ;)
The hard things are not fixed in stone; they are challenges to be solved. That's why we bother with society; if it weren't worth our time to take on the hard stuff, we'd have stuck with hunter-gatherer life style because it's a lot structurally simpler.