At a past company, we (incorrectly) assumed we'd always have to support an on-prem deployment capability, so we decided to build our own virtual appliance (assuming we'll do actual hardware one day). That appliance cluster had to do a bunch of heavy lifting to provide a bunch of services, and we had to do it all ourselves. We even had a Cassandra-like DB cluster, which we stupidly tried to do using Scylla-DB (Scylla is amazing now, but it was just getting started at the time, and while it was super fast even then, it was not stable or reliable enough at the time). To add insult to injury, we only did a single on-prem deployment ever, and that customer never actually converted to a paying one...

If I could do it all again, I would have gone cloud-native (or at least leveraged K8s), and I'd use as many managed cloud services as humanly possible. At a later gig - we did just that, and we very very rarely had even 1% of the infra struggles we had with the solution I described above.

Nowadays, my basic advice is to always buy the best possible service when you start out, and only start to think about replacing it with DIY services when you have enough scale to pay talented engineers a salary to build AND support replacing it - and even then, the potential loss of focus and velocity might still make this a bad idea. There's a reason Netflix is still on AWS.