Because the two main reasons that a company would be running an on-premise cluster is (a) security and (b) cost. Both of which conflict with running a hybrid model because it is extremely complex and expensive to do so.
This technique is called “cloud bursting” and is done often enough to have docs on Azure: https://azure.microsoft.com/en-us/resources/cloud-computing-...
More generally, hybrid on/off prem (or multi-cloud) is a top level feature of GCP’s Anthos.
You’re not wrong about your two main reasons, and it’s true that it’s hard to wire up. But you definitely can wire up hybrid workloads in some cases.
The obvious one would be stateless batch compute like image/video processing; “I prefer to re-encode my videos on prem but can burst to the cloud if I get a traffic spike”. This might end up being cheaper than either of all-on-prem with overcapacity, or all off-prem with auto scaling.
I agree it’s not likely to be useful for something where security is your reason for going on-prem.
For security concerns, you would probably be selective about which services and workloads you move to cloud compute, possibly preferring to offload internal services like CI before customer data.
Regarding cost, are you implying that there's large monetary cost to preserving the ability to spin up cloud workers in an emergency, or just that once you do it's expensive per day to keep them running? Presumably if you're running an on-prem cluster for cost reasons, you would work to scale it up quickly after an unexpected sustained demand spike, and the cloud supplement would be temporary. If you're getting "unexpected" spikes every week, you need to work harder at forecasting.
In any case, I suspect today that people running on-prem clusters provision them with the intent of being able to handle a load growth and keep weekly average utilization at <60% during the spike. Investing in building infrastructure that keeps working smoothly when weekly average load is 90-100% means you can provision a smaller cluster either way.
In 2007 I was working for a webhostet / nascent iaas provider, and we had workloads doing exactly that.
Specific example would be Celtic FC who had a baseline of dedicated, but would scale into our VMs during events, i.e. uefa cup games.