Basically, there are only so many hours of outsourceable stuff in your day. Once you've made the trade for cooking (eating out/chef/stay-home-spouse), cleaning (maid/stay-home-spouse/etc), errands, scheduling, minor research, and other miscellany (secretary), local transport (owning a car) there isn't much more to do.
If you really value your time, you can fly a private jet at the cost of maybe a few thousand dollars per hour saved, I guess.
And Warren Buffet, even in his frugality, does all of these things.