Finally, someone explained it to me from a different perspective: If my partner didn't want to leave then I was the one with the problem, not him. Instead of looking at what I was missing by him being there (a fact I couldn't get around), I should look at what I could be doing on my own without him. I talked to my partner about it and we agreed I would leave and keep some of my shares instead of a buy-out which the company couldn't afford at the time.
8 years later he's still running the company the way it was after I left. I sold my shares to a new partner for a price that was many times what I would have received if I was bought out when I left. In the past years I've been able to build a company like I envisioned the first time, sold it and am now in the process of repeating that for a second time.
Looking back I've come much further than I would have if I stayed at the company. My former partner found the perfect balance between his company and his family life, and we've been closer friends for the past years than we had been before working together.