I chose to go to grad school, and along the way wound up working for a healthcare informatics startup. Great team, I trusted them a lot, even when they had a major product pivot. Last fall, I had to make the choice, wait for a spring offer to join them full time (yes, please and thank you) and reject other fall offers, or "play it safe." I took the handshake agreement and it wound up falling through. Am I pissed? Yes. Am I glad it happened? Well, I got a great learning lesson out of it, and am now doing my own startup in data analytics, commercializing a side project I did during grad school.
What I'm trying to get at is that yes, you will always regret the "what ifs?" However, be rational about the team, the product, and the vision, and the opportunity cost of putting your education on hold. Yes, you can always go back, but I've had friends do it, and it is hard(er). I'm happy to talk more about this or follow-up, since I know from semi-first-hand experience a bit of what you're going through. You have a lot of options on the table, but just look long-term and be true to yourself, more than anything else.