I didn't say it was my only reason. I said "This is a huge reason...". There are quite a few others as well. I didn't list them, because they're irrelevant to the "Ask HN" question, and are therefore off-topic.
I actually recognize that line of reasoning myself. But since losing weight and working on myself (e.g. going to a therapist) I feel much better.
My point is: There was bigger issues at play for me when I didn't want to work from an office...