If you're a poor startup, the objective is to solve a problem. You can solve a problem without great design, just as you can solve a problem without great programming, or great copywriting, or great hosting, or any of the elements of building and running a startup. The only thing a startup has to do exceptionally well, I think, is to solve the problem it sets out to solve.
I know some designers who have started startups and naturally, their pages are beautiful. I could make a similar argument about how development/architecture should come first. Their apps will never scale, and if they'd had a good solid architect in at the start, we could have solved all their IT issues. That may or may not be true of course, but it's easy for me to make the statement that I could have helped and I'm probably right more than I'm not (just as you are) -- but that misses the point that except where funded, startups should focus on the things they can do well until they can prove the market or figure out if the idea has legs. Once you've proven that there's a benefit, or a customer base, or a way to make money, then you can start to pay money for things to make something more well-rounded.
Note: I'm re-reading this before I submit, and I can't tell if I sound like a giant prick or not, so please be aware that wasn't my intent. Your statement is of course valid, and I would personally love for startups to be better in UI/UX; I just wanted to posit that startups can't be good at everything and still be quickly built / cheaply built / etc.