Obviously, you want a sane legacy for the future but unless you already have hit that sweet product-market fit spot it is likely that you will spend a non-trivial amount of time optimizing (prematurely) for something that's in fact orthogonal to what you actually need.
The trick, as usual, is to balance engineering and business depending on which stage your at.