Wrong lesson.
Don't put secret keys in your repository.
Someone getting a copy of your code should be a big annoyance at worst.
Don't put secret keys in your repository.
Someone getting a copy of your code should be a big annoyance at worst.
The right lesson is: Know where your secret keys are and take the appropriate steps to secure them. Whether that's in the codebase, a properties/ini/conf/whatever file, environment variables, whatever - know where they are and make sure you understand possible threats against them.
This story could just as easily have been written about how easy it is to download ALL_THE_SECRETS.txt. Don't feel smugly secure just because you don't store passwords in git.
1. Make a list of everything that absolutely positively cannot live without this data/access/permissions/etc.
2. Put the data somewhere where absolutely nothing whatsoever can ever read it (except root).
3. Figure out what one single change will resolve #2 so that the things in #1 can happen without any other things gaining access.
If you don't do #1, you don't understand your requirements/applications. If you don't do #2, then your data is probably vulnerable through some other mechanism. If you can't do #3 then you probably need to change something else (e.g. stop running all processes as the same user, stop running all services on the same box, stop trusting users, set up more granular sudoers rules, etc).
What I find is that when you come up with an idea for #3, and then come up with a list of side effects, you can actually find a lot of the kinds of issues I mentioned above, for example where the public website CMS (as 'daemon') and the accounting backend (as 'daemon') both have access to the same resources, and thus someone gaining access to the CMS can get the accounting DB user/pass and get access to your transaction records, user database, etc.
It's important to know where your keys are, but it's also important to not store your keys in certain ways that are easily overlooked.
A lesson of "don't put secret keys inside the web root" is also useful.
But a lesson of "know where your keys are and secure them" is a bit too short-sighted. You don't just want them to be secure right now, you want the mechanisms keeping them secure to be mistake-resistant.
Don't put them in the code, even if you promise to be super careful.
DB_USER=scott DB_PASSWORD=b3withm3pl3aze /usr/bin/python webapp.py
That could be troublesome if an attacker figured out a way to run remote commands on your server even as an unprivileged user.
pikachu@POKEMONGYM ~ $ sleep 99 &
[1] 21340
pikachu@POKEMONGYM ~ $ cat /proc/21340/environ(I actually gave the wrong example in my previous comment. While it is true that giving the ENV on cmdline will show up in ps eaux, the more appropriate example is what I just explained in this comment.)
Now, if an attacker manages to get root access then it's game over[1]. That just shouldn't happen. But nobody should be running their webserver as root. So, whatever that user is should be low-powered with only enough privileges to start the webserver & bind port 8080 (and use iptables or whatever to reroute connections to port 80 --> 8080) and the whole setup should be designed that this account won't be able to escalate things further if someone got a bash shell to it.
______
1. You should at least have some way of detecting that it happened and consider all data & files compromised and just wipe the whole machine & start over. Or take that machine offline for investigation into what happened and put a fresh new one in its place.
It would be kind of weird to include that considering how argv works.
What are your other 11 objections?
/srv/www/domain.com/public_html/
|--------->/private/
|--------->/logs/
|--------->/tmp/
It's specifically for storing things like configuration files.
I think this should be standard practice.
Disclaimer: I worked on it.
It's relatively common to provision secrets with configuration management software like Chef/puppet/ansible/etc using, e.g. Chef's encrypted data bags.
Another slightly heavier-weight solution with some nice properties is to use a credential broker such as Vault: https://www.vaultproject.io/
I typically handle this by versioning a `config.example` file, which includes all the necessary config keys an application expects. The example file defaults these attrs to various strings meant to show they are examples only. I include instructions to copy the `config.example` to a `config.yml` (or some other appropriate extension), and replace the values as necessary. The `config.yml` file is specifically excluded in the `.gitignore` file. The application will only load the `config.yml` file when started, so I also ensure to raise a descriptive error informing team members when they are missing a local `config.yml`.
This allows the `config.example` to also serve as a self-documenting config for the application, as comments can be included that identify and explain each of the config keys and their purposes.
Usually the base config is in VCS but without user/password/db strings. We then manually configure the file with the encrypted strings on the server (usually with the machine name in the filename so that we can use hostname in code to find it and makes it clear the file is machine specific). Not all tools make this easy though and only works if you can add your own code in between. Also prefer files to environment as the files can be locked down easier in my opinion and more obvious what is going on.
I like some of the other solutions that are using encrypted strings but with a keystore server and may consider for the future if they support both windows and linux.
There's a variety of mechanisms for loading this into your environment.
1. Create a JSON file containing encrypted secrets (DB pass, etc.)
2. Upload the file to a secure S3 bucket with fine-tuned permissions and server-side encryption
3. For the instance that is launching, include the permission in the IAM role that allows it to "S3:GetObject" on the specific JSON file you uploaded.
4. Deliver decryption keys to the app in some manner (Chef, Ansible, etc.)
5. When the app starts, it downloads the JSON file and loads the environment variables.
I wrote a blog post[1] about this with more detailed info and an NPM module if you use Node.js.
[1] http://blog.matthewdfuller.com/2015/01/using-iam-roles-and-s...
I have a question about redundancy or "What happens if your gatekeper EC2 instance goes down"? If you have multiple gatekeepers could they be set up this way:
- let's say you have five different web apps using a gatekeeper to hold their secrets
- let's say you have n gatekeepers (let's say 3) and each of the apps knows the address of all three gatekeepers.
- If the primary gatekeeper is unreachable, all five apps would try to contact the secondary gatekeeper, but that gatekeeper would only (ever) respond in the event that the secondary gatekeeper also found the primary gatekeeper unreachable.
It's like a sleeper cell - at any given moment you have multiple replacement gatekeepers ready and waiting to serve, but each of them is unable to respond unless the one above it in the list stops responding. In this way you could lose gatekeepers (even permanently) and build a little bit of resilience into the apps depending on it while you're able to sort out what happened and restore normal behaviour.
Is this a good idea?
Another way is a second file that overrides settings as needed. Although I have found that to be less maintainable if the configuration file changes. That file should be somewhere entirely out of the VC tree.
Either way, the file must be placed in a directory that is not served by the web server.
/include
and
/public
are traditional. Only /public is exposed by the web server.
use git crypt[1][2]
use git-dir and work-tree options/env vars
[2] though you have to remember to git crypt lock