I had to pivot a startup far more than 3 times to get to success. The key is really how fast it gets to product market fit.
Before that, startups need to build very cheaply and crudely. E.g. I wouldn't even build a database unless I'm certain I need one - in many prototypes, hardcoded data or dummy data works too. Once you get the first $1000, you can start scaling.
I've seen multimillion dollar companies start with an annual revenue of $10k. Startups always start slow.
To answer the question, you should have a good gutfeel for it. If you think the leadership is strong and the product is something people want, stay. If you're just in it because you think the market could be big, but your heart isn't into it, then leave.
Startups thrive on the outliers. There's not much process to it.