I guess the same as people have always done. You have more server rooms on ideally different networks and grids. That can be a rented room/ rack in a bigger DC, your own if you have a big factory with subsidiaries etc.
The IAM etc. will probably done by a combination of technical on organizational measures. You will have certain people doing certain things at least before the solution is ready for IaC. People will create roles, accounts, accesses and such. With networking gear that can be still tricky to implement, with virtualization solutions that is easier today. For databases etc. you can create accounts in them too.
Of course, K8s and similar make these things more formalized/ transferable. However there is a lot of stuff before you can deploy that.
People forget however that even if you have hundreds of servers you are tiny compared to the cloud providers. You don't have to have the same breadth and depth of offering. So while you need more baby sitting of hardware, probably will not get nearly as good deals on hardware as the big providers do, you will save their considerable margins. Also, they actually have some of the same expenses too - if a harddrive goes bad they will still swap them basically the same as you do. Big cloud providers will not get substantially different energy pricing than what e.g. a steel foundry would get.
By hosting things on premise or in a nearby datacenter(s) you can shave off a lot of latency too. Some machinery likes to store a lot of data and shaving off latency will decrease your need for thick router buffers because you will not have such a big Bandwidth Delay Product and will achieve the same speeds with much smaller buffers.
Building stuff on premise just for you makes some things easier too. Even if you loose some credentials usually you can just hard-reset the equipment as the last resort. There will be no credit card blocking that would affect the operations. If you are less strict with security it will usually matter much less - you are not sharing the hardware with unknown parties and all people that touch it have a contract with the company. So usually everybody want the company to succeed to get the paycheck. You build a deeper know how and can do some optimizations the cloud providers cannot do because you don't have to be general.