> Rotate secrets automatically every few hours
How is this helpful? To automate it means there is another system that could be attacked and it‘s a valuable one as it manages all the secrets, or not? What‘s the story?
How is this helpful? To automate it means there is another system that could be attacked and it‘s a valuable one as it manages all the secrets, or not? What‘s the story?
If you automate this and run it on an automated schedule < 30 days then it is pretty likely that it won't be causing downtime unexpectedly, that you'll have monitoring in place to make sure it actually gets done, that, even if you forget to trigger it for a specific reason (e.g., aforesaid person leaves the organization) it will happen within a reasonable period of time.
In terms of securing such a system... you need to make sure that you separate the system into appropriate pieces with limited access. So, for example, you want a job that is run with an account that only has access to rotate the credentials. It can't use them for anything, just rotate them. Services that consume those credentials should not be able to update them, just use them. You can then ensure that the process that rotates credentials executes in a highly locked-down part of your infrastructure.
Indeed, automating this process also encourages you to create processes with limited access, rather than relying on administrators who have so many responsibilities, you probably just throw them in the equivalent of wide-open sudoers file and call it a day.
It sounds complicated, but if you have decent abstractions, this kind of stuff is actually pretty easy to accomplish.
I'd be interested in seeing any end-to-end examples of how people are doing this in practice.
For example, suppose you're maintaining a SaaS application and you have a private key to access some third party API that certain parts of your back end code need. How do you automate this process, so you change your private key on a regular schedule and update all affected hosts so your application code picks up the new one?
Ideally this needs to avoid introducing risks like a single point of failure, a new attack surface, or the possibility of losing access to the API altogether if something goes wrong. Assuming the old key is immediately invalidated when you request a new one via some API, you also need a real time way of looking up the current active key from any of your application hosts when they need it, again without creating single points of failure, etc.
No doubt this could be done with enough work, but it doesn't feel like a trivial problem.
That doesn't solve your problem, but it means that you should take this complaint to whatever API service offering you are using.
1. The system that gives a service to the public is publicly accessible by default. The system that rotates its credentials it doesn't need to be; it can sit behind your firewall listening only on one port for ssh, with firewall rules allowing access only from a bastion host.
2. The credential rotator also connects to your server and drops credentials into it; you don't call from your server to the credentials rotator, because see Point 1 above. This limits the attack surface.
You're right that everything is potentially vulnerable, and so would the credential rotator system. However, by rotating secrets this way, you shift the locus of security to a smaller place that you can defend much better.
Same goes to 2 apps connecting with a shared secret, making both of them change it wont expose any more components but will add additional layer of security (like google authenticator)
During a breach, if each service gets their own secrets, it becomes easier to trace the entrypoints and which secret go compromised. Once the system is closed again the attacker automatically looses access to everything after 4h.
Credential rotation processes are yet another layer of “defense in depth” — the more layers you have, the more secure you can be.
As you rightly point out, it adds complexity. Having a credential generator+rotator means you are more resilient to the fallibility of humans choosing bad credentials or too busy/lazy to do the task.