"Work smarter". If other people are putting in a 40 hour week, then you can automate yourself and increase efficiency, then do a 20 hour week and do stuff you want to do rather than must do. In my case that means spending time with my family.
Elon does a miserable job of family by the looks.
There is no need to compare your own family life with what you expect others' family life should look like.
But this kind of work habit will affect your personal life, and Elon has already gone through 2 divorces. This is perfectly relevant to the discussion.
The quote about working 100 hours a week in that case does not apply to you at all. And that's fine too!