Yes of course, the secrets have to be isolated from the VMs, preferably at the network gateway and not on the same KVM host. But as far as I know there is no safer containerization technology than a VM, and the only way to be more secure would be to have a physical computer per agent process? A KVM does not use the host kernel and user space, and provides hardware level isolation (the CPU hypervisor etc), that's the point. I think that once you are inside of vm, just using firejail is the better approach, since you have more direct control over the OS level controls that are being leveraged by various container solutions anyways. At any rate, that's what I did.
If you can't isolate a computer on your network, you probably can't isolate your network from the internet, so it's an irrelevant exercise at that point. And yes, probably you can't do any of those things and any frontier model could technically hack your network, but I don't think there's a better way to do it?