Show HN: SecretCrypt – Keeping secrets in plain sight | Hacker News Reader

Show HN: SecretCrypt – Keeping secrets in plain sight

zemanta.github.io

75 pointsnsaje35 comments

> Since the configuration file is kept in the same repository as the code, configuration options or secrets can easily be changed or added by developers themselves.

This is terrible advice. Code and config should be separate. Otherwise you can't deploy the same code to a different environment.

We deploy code to 3 datacenters so we simply have 3 config files in the repository. Deploy script then decides to link the correct config for the environment.

If you want to run local environment you can simply create your own configuration.

I don't really see what you gain by moving configuration files into a separate repository.

> If you want to run local environment you can simply create your own configuration.

Externalizing config forces you to cleanly handle local environments as well. There's nothing special about them.

I generally recommend having an env.example file listing out required environment variables with sensible local defaults. This eases onboarding a new developer or even an existing developer on a new machine.

> I don't really see what you gain by moving configuration files into a separate repository.

Because config and code should be kept separate. Updating the database that your production app is using is not a code change. It's a config change.

Setting up a new CI environment should not require pushing new code.

If you have a production issue in version X, separating code and config allows you to reproduce it using an isolated database/mq/foobar service. It's exactly the same code.

I'm sorry but I really don't understand how having env.production.toml commited in the same repository as code makes anything that you said impossible/more difficult.

I can always grab the build from production and run it locally with the modified config.

> env.production.toml

Which production env? Which datacenter? Which database cluster? &c

The deployment should take care of generating configs, they shouldn't be hardcoded or stored anywhere imho.

I mentioned earlier we have one config per environment. So for example env.use1.toml, env.use2.toml and env.usw1.toml. Then one more for ci and one example for local development.

Sure deployment can generate configs but it has to take it from somwhere. So you need a database preferably with versioning and encryption. It's also nice if you know which settings are used in specific builds so you can go back and debug past production issues.

Of course for larger teams and deployments it makes sense to develop and maintain this infrastructure. But since we already have git, why not just use it.

That's pretty standard and accepted. Don't know what gp is talking about.

Unless your code is like 99% of all the code I've seen ,where you only deploy to a handful of locations, and it's really handy to just say '-profile=stage'. Besides, including config in the build does not preclude being able to pass it in as well.

Yup. Storing config and code together is a violation of the Twelve-Factor App methodology. https://12factor.net/config

I keep hearing this "violation of Twelve-Factor" statement.

The Twelve-Factor Methodology is a sensible set of guidelines/defaults for building a modern app, as written by Heroku.

Don't blindly follow it As It Was Written without thinking. Seriously, do what works. Use your brain. Using this statement as an argument against other methods is sending the wrong message.

The linked part about config makes sense. It's a specific instance of a general concept that shows up in hardware and software engineering. That's that things that reusable things that don't change should be separated from those that change often.

In this case, we have code and [configuration] data. The data can and will vary considerably while reusing the same code. They should be separate so data changes don't require recompilation. Further, any change of code can change its behavior. Better to have code that's already vetted via review and testing to handle specific types of input data. That way you can keep changing the data without re-assessing the code. There's also a minor benefit in efficiency when you can re-use binaries already created and integration tested while just generating new configuration data. Those minor things can add up if you're constrained in CPU/memory.

> data can and will vary considerably while reusing the same code.

This is the complete opposite of my experience. We deploy new code to production 10 times per day and change configuration a couple of times per month.

What do you keep in configuration that changes this often?

Well, it depends on internal projects vs external ones. Internal ones might work as you described. External ones will have many more deployments than code changes for a given module.

Agreed. I always worked with internal project so it didn't cross my mind.

I agree with everything you say! Notice I'm not saying that the 12 Factor App is incorrect or bad - I think it's quite the opposite. It just concerns me that people treat it as gospel, and quote it like it's some holy work that cannot be disagreed with.

what's your counter argument/proposal ?

They are just guidelines. "violating them" means absolutely nothing so claiming so adds nothing to the conversation.

This is exactly what I meant.

Here's my interpretation of the link:

https://news.ycombinator.com/item?id=12344971

Is this methodology accepted as state of the art or used by the author of this article? I don't see why breaking this particular set of rules is an issue.

You can have per environment config files

Using KMS is a good idea, but I'm not really sure what this package gives you - just a simple abstraction / cmdline to deal with it?

This would be better if it could use the aliases directly, so you could have one config across N environments and separate them by AWS keyspace.

Having to embed the full KMS path for each key gains you the secret management they claim (which is a good thing) but sacrifices ease of use. That said, aliases wouldn't help with missing secrets or misconfiguration across environments, and its a lot easier to audit string-for-string to match your KMS store, so either approach has its pluses and minuses.

How is this different from/better than Stack Exchange's blackbox[0] which doesn't require a third-party service (just plain-ole gpg) and is written in bash?

[0] https://github.com/StackExchange/blackbox

P.S.: I think the image looks aesthetically pleasing, but why is it there? It's a scaled-down 1,600x680px image that costs me 140KB and doesn't add anything to the article; what's worse is that it's not even a nice banner image, it's just smack dab in the middle of the article.

That's the company logo, so it's the ad that makes this blog post possible.

> image that costs me 140KB

What year is it?

That's besides the point. Wasted bandwidth should be avoided.

So... storing your secrets in AWS then... and not in plain sight.

Yeah, that wording is odd.

Storing your secrets in plain sight*

* as long as plain sight includes a remote service call at some point

I like this design. Infrastructure as code. Store your config data in you repo. Screw 12 factors.

You can do this with the inhouse AWS tools, awscli and boto3 for python This was for use within a python lambda function so i used the secrets in a seperate file, but no loss of generality here.

* Create your keys in KMS via Web UI or otherwise * encrypt your secrets before commit aws kms encrypt --key-id alias/TokenKey --plaintext fileb://unencrypted_token --output text --query CiphertextBlob > encrypted_token

Decrypt the token from your python lambda function with boto3

kms = boto3.client('kms') token = kms.decrypt(CiphertextBlob=base64.b64decode(token_encrypted))['Plaintext'].decode('ascii')

The blob from KMS contains the appropriate fields for decryption from their service. Give the lambda role rights to decrypt with the key.

It should be made clear that this requires AWS KMS, and for automatic decryption, EC2 (so that the instances can be associated with an IAM role that has key decryption permission).

I think that's pretty clear in the article that's linked. It explicitly mentions AWS KMS and its reliance on it and how IAM roles are used to grant access to a secret.

It also states that KMS isn't a requirement, you can use Vault too.

Repository: https://github.com/Zemanta/go-secretcrypt

What happens if Amazon's KMS service is down - does that break the app or can you operate ok without it ?

We decided to decrypt secrets on service startup so if KMS is down during the deploy we can stop the deploy after first server fails to start. Not perfect but good enough for our use case.

In case of catastrophic KMS failure we can always manually replace secrets with plaintext and revoke them afterwards.

The "write-only" feature could be implemented without a third-party with a keypair.

Reply on news.ycombinator.com