Having worked in 3 startups that were AWS-first, I can say that you've learned the completely wrong lessons from your time at your cloud providers.

Building on AWS has provided scale, security, and redundancy at a substantially lower cost than doing any on-prem solution (except for a shitty one strung together with lowendbox machines).

The combined AWS bill for the three startups is less than the cost of an F5, even on a non-inflation adjusted basis.

The cloud doesn't mean that you can be totally clueless. I've had experience in HA/scalability/redundancy/deployment/development/networking/etc. It means that if you do know what you're doing you can deliver a scalable HA solution at a ridiculously lower price point than a DIY solution using bare iron and colo.