Solo founder? You are by default "remote", unless you're a butts-in-seats type of manager. Two co-founders either both technical or both business? Probably remote. One technical, one business? Maybe consider colocating, depending on trust levels, communication styles, etc. Three co-founders? The "maybe" gets stronger depending on the communication overhead.
Communication style and skills: the better your team is at asynchronous written communication, the easier fully remote work will be.
Cultural diversity: Some kinds of cultural diversity require better communication skills to bridge successfully, for example guilt vs. shame cultures. This is not necessarily easier to do in person, but it may be easier to recognize a problem (empathy is generally easier f2f).
Introverts will usually work better remotely (possible exception: if they can have a private office), while extroverts may wither away or just find it harder to stay engaged without in-person contact.
Etc.
There are so many factors, and so many caveats, that it really does boil down to "it depends".