Well, we can complain about it, but it's a fact of life. It's almost like complaining that gravity drags us down :-)
Plus, for a lot of people does give them... something to do. A purpose, an activity, a place where they can use their skills and use their intellect.
It's hard to do this alone if everyone else is working. I guess you could try mingling with the rich people and join charities?
But I do try to exercise my empathy and recognize for some endeavors it's is useful to collaborate. I just wish that people wouldn't attribute intrinsic necessity of physically close collaboration where it is not warranted.
My purpose or activity, even after I'm financially independent will be writing software, something I can do without having to be in close contact with people I am working with, and I in fact am much more productive when I'm not.
And honestly, even at 40 hours a week, I still have a lot of time in which I've formed my best social relationships. My coworkers are nice people, I just have shockingly little in common with them outside of work.