The issue is that there are very few jobs where you can basically ignore the rest of the world (or even the rest of the world in nearby timezones). If you're a solo writer or programmer sure. But if you work with other people, not wanting to have meetings before 2pm is often a problem. (As would someone who wanted to work midnight to 8am.)

Even jobs that are quite flexible are going to want some overlap time and, even if an atypical schedule allows for a few hours of overlap, the reality is that people will get tired of always having to schedule around your preferences.