As someone who bootstrapped a "startup", there are plenty -- PLENTY -- of non-SaaS solutions that:
1) are perpetually licensed. You don't have to worry that they'll disappear on you, or jack up their prices.
2) are cheaper. far, far cheaper.
3) are just as effective.
4) enable growth by:
4a) Being extensible (server-side code execution enables a lot).
4b) Being economical (SaaS chews up a runway like nobody's business).
4c) Building valuable (in the business agility sense) internal expertise.
5) don't incur the overhead of managing accounts for people across a different provider for every service. If you're growing fast, having a single account switch to flip makes a real difference to new-hire integration overhead and effectiveness.
Hard dependencies on costly software subscriptions is rarely a net win for a startup.