How far do you go, how do you use the private key to sign something if you can't keep it anywhere?
You never have the private key, only the ability to ask something to encrypt/sign something
This is a hard tradeoff between availability and compliance. If the cloud service goes down or you have an internet issue, you would lose the ability to sign any new tokens. This is a fairly fundamental aspect of infrastructure so it's worth considering if you absolutely must put it across the wire.
It crosses from everyone has the keys like in this example, to centralising a signing service using just software, or using something like KMS or CloudHSM, or YubiHSM, or going big and getting a HA Luna (or similar) HSM setup.