The cost differential in my experience is generally higher on the low end. Once you get into the hundreds of k per month in cloud fees you can get steep discounts, and some of the gap starts to close.

You also need devops resources to manage a cloud setup - most of my jobs in recent years have involved exactly that, and it's not appreciably cheaper than managing "raw" hardware resources these days (ever since IPMI etc. became the norm); for some time I did consulting and provided devops services on retainer. The amount of time spent managing actual hardware was a rounding error of the overall devops cost for those of my clients relying on colo's or managed servers.

Since many colo providers also provide managed servers and cloud services, the flexibility is also not really an issue - you can start with cloud services, rent managed servers as you know what your base load is like, and can tie them into the same setup, and do the same with colo services if/when it's cheaper (the tradeoff between managed and colo is trickier than with cloud - the cost differential there is often dominated by realestate costs where the relevant data centres are; if you're somewhere expensive, managed servers elsewhere will often be cheaper than colo near you)