A good number of "world changers" prioritize their work over everything else in their lives, including themselves and the people around them. They expect their team to do nothing less, so if he did say that, I'm not exactly surprised.
This wouldn't work for me, and I'd imagine it wouldn't work for a lot of people who prioritize their families over whatever project they may be working on. I still like Musk and the things he does, but I have no desire to work for him if that means working 24/7 like he does.