For a large company with accountants they can do the accounting math, and a rack or two of servers is expensive - it need real estate, trained employees, managers for those employees, HVAC, power, backup servers, backup power, .... This adds up faster than on-prem advocates think. Many of those costs are things that work better in data center sized warehouse than a smaller company and so cloud can often be the a lower price once you really add up the costs. This is particularly true if you can share cloud resources in some way.
On-prem only makes sense for really large organizations, exactly the ones that can negotiate with cloud providers.
What I do expect to see eventually is a large movement towards commodity IaaS providers. And those tend to be middle-sized and not design their own hardware, so they act like the on-premises market. (But I can't say I'm seeing any movement here either.)