On the question of whether you should leave, I think it may be helpful to think of this startup as a high school love. Imagine you're 15 or so, and just met the guy/girl of your dreams, and are infatuated. You picture yourselves being together forever, getting married, settling down and living long happy lives together. You want to drop out of high school so you can get a job so you can move in together. It'll be amazing!
Most adults will look at that story and know the answer. You're just infatuated because it's your first love. It's probably not going to work out anyway, once you do live together. There will be more loves and more opportunities.
Now to give you the other perspective. I know a bunch of people who dropped out of college to be founders or founder-ish roles. Some of them founded or joined incredible startups (Stripe is a great example), have made a meaningful impact on the world (and a bunch of money too, though it probably matters less to them).
Of course, it's very hard for you to tell which of those situations this is...