Moving will result in saving more money, and that will further your goal of financial independence.

If the company goes down you can always move back to NYC.

> If I do, should I buy a house there or rent? My company is a startup.

Uh, why is that even a question? Just rent. If you do have money for a down payment then save it for when you're really getting settled.