Hacker News Reader

Comment by politelemon | Hacker News Reader

If I'm understanding correctly, a whole bunch of credentials, like IAMs, DB passwords, Steam keys, and MailChimp keys were lying around in S3 buckets.

But I don't understand the use case, what would be the purpose of uploading those details into S3 buckets? Or I suppose I'm trying to reverse engineer the situation where the dev/ops team decided to do this.

View on HN

S3 keeps secrets out of source code, so you at least don't have to purge git history and can lock access down to "internal developers", and can relatively easily rotate the creds, just find everything in the creds bucket (instead of searching all your code).

Handling of secrets has gone through many rapid iterations in the cloud lately since around 2013.

For AWS: In Source. In a magic file that lives on build machine. In S3 with crypto at rest that you can pull when you boot your machine, or dynamo, or DB, just one boot password or IAM role to get you access to the rest. Then in Envvars for the service. Then Secrets manager / SSM Parameter store, more recently.

Various organizations and pieces of software are somewhere along this curve. And the less cared for this software is (or even known about, people forget software), the further back on the curve it likely is.

Beyond the above methods that is a more constant rotation behavior similar to Hashi Vault using SSM/Secrets manager. And a drive to require all systems to use constantly rotating credentials (no static creds). I'm not sure what comes after that.

However what system you use is highly dependent on your organizational maturity and internal threat model.

Rather than use a password manager, or credential store, or some other secure way to keep these credentials safe while providing access to internal developers for development purposes, they put them on S3.

Here's an example I have seen: - env file is needed for development to run a service on development machine and to access the staging deployment - the credentials in the env file aren't per-developer because that requires work to setup accounts for every developer with the staging hosting service - so make a copy of the credentials, put them in an env file on the NAS - NAS isn't available from home or from other network locations - so make a copy of the env file in the cloud

If the S3 bucket hadn't been public they probably would have been fine.

One can only speculate but I can't imagine how many companies will avoid investing in security here, because they think that the secrets in their git repos and S3 buckets are perfectly safe, and they allow some people to skip 2FA because it's too inconvenient for them, and some people have root access on AWS because it's easier, etc. Maybe even giving the job to people who don't have much experience in the field and are still learning how to set up things in the cloud.

A publicly accessible S3 bucket suggests that someone mistakenly thought it was private, even by obscurity.

Also, if you don't have a public access block in place, a private bucket can contain public files! Even if you can't list the files in the bucket, there are tools which try to guess common file names from guessed bucket names e.g. sega-secret-sauce.s3.amazonaws.com/.env - if someone uploaded a file there without setting the ACL correctly there could be an unprotected file in the private bucket.

Why they used users in the first place I don't know, but for IAM credentials - I've seen people using Terraform to generate the users and access keys, and storing the access keys in the terraform state (you can't access secret keys after they are generated), and the entire state of Terraform is typically stored in something like an S3 bucket.

It's definitely not a great practice, but still it's done.

Reply on news.ycombinator.com