Dockerfile Security Best Practices
cloudberry.engineering
cloudberry.engineering
I think it's counterintuitive, but I learned the hard way that 3306:3306 will automatically add a rule to open the firewall on linux and make MySQL publicly accessible.
Is that true? 3306:3306 would bind the port on all interfaces, but I was under the assumption you'd have to explicitly enable firewall port 3306 for the machine to accept traffic to port 3306 from outside of your machine.
I'll have to test that.
When an IP packet related to your container arrives (<host ip>:<host port>):
- docker rewrites it to target <your container's ip address>:<your container's port> (NAT table, chain PREROUTING delegates to DOCKER chain with the relevant DNAT entry)
- since the IP does not match your host, the packet is forwarded (there's a relevant routing entry pointing to docker's virtual interface)
- the first thing you encounter in the FORWARD chain of the FILTER table is a few jumps to the docker-related chains, DOCKER chain in particular accepts all packets destined to <your container's ip address>:<your container's port>
So a few takeaways:
- your standard firewall might not be involved because its chains are plugged in after the docker chains in the FORWARD chain (e.g. ufw under Ubuntu)
- if the above is true and you want your firewall to matter, you have to add stuff to DOCKER-USER chain in the FILTER table
- at that point the host port and IP doesn't matter since it's already been mapped in the NAT table's PREROUTE chain at the beginning of processing - write your firewall rules to address specific containers
There's no way this is true. This completely defeats the purpose of a firewall.
If it is happening, then it's Docker doing this - not the linux firewall that comes stock with most distribution (iptables and the like). They would never simply add rules to the chains just because something was listening on that port.
Really, the best security advice for using Docker is to not use Docker. Unfortunately, there aren't very many "hold your hand" alternatives available. Aside from LXD and that family of technologies, which are criminally underused.
It's not so much that they explicitly open a firewall rule, as that they take a networking path that isn't really covered by traditional firewalls.
Another way of viewing it is that Docker "is" your firewall for your container workloads, and that adding a port-forward is equivalent to adding a firewall rule. Of course, that doesn't change that public-by-default is a bad default.
Terrible practice, in my opinion. Docker shouldn't be touching firewall stuff.
For most container purposes, host networking and the default process namespace is absolutely fine, and reduces a lot of problems with interacting with containerized apps. 95% of the use case of containers is effectively just a chroot wrapper. If you need more features, this should be optional. This would also make rootless federated containerized apps just work. But nobody wants to go back to incremental features if Docker gives them everything at once.
It's not as powerful as the later tables in the chain (see https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilte... ) but a lot more robust.
My current policy is to set `"iptables": false` in Docker's `daemon.json` on any public machine. I don't understand why this isn't the default.
If you don't muck with iptables then you need a (slow) userspace proxy to expose your pods. That also means losing things like the source IP address for any incoming connections.
I do see that RemoteAddr is from a private IP range. Luckily I'm not using this information anywhere, but good to know.
It’s easy to blame Docker but I’ve seen this failure mode happen many times over the years - even experienced admins make mistakes. As always, defense in depth and continuous monitoring trumps raging about software.
The default is terrible. It got into 1.0 and just got stuck there :(
Changing defaults on such widely used software is, unfortunately, hard.
For whatever reason Docker prepends rules. There are things you can do to add your own filtering (Docker forwards to the "DOCKER-USER" chain where you can put your rules), but it requires people to know what is happening I order to use it securely.
It would be really nice to be in a more secure situation by default... open to suggestions and contributions for Docker 21.
The nice thing is admins can already define a default value that the eninge will use to bind to (when no address is specified on -p). Warning can point users to that setting.
This also raises the possibility of different security profiles like dev, prod, etc.
A default Docker install would be documented as being for development, and you run "docker secure" to change that for other environments.
Is there official documentation that tells users to set the default bind address as a best practice?
I wasn't thinking so much of just changing one setting, but rather having a way to easily reconfigure an installation to set multiple settings to improve security.
In addition, elevating this to the status of a command and documenting it as a best practice helps spread awareness.
I like Godel_unicode's suggestion of logging and that could probably done in a stronger manner if there was some point (post-install, maybe starting a container) where it checked the existing rules and used a more prominent warning when there are existing rules which would prevent a container which would be reachable now from being reachable in the future. Given how widely Docker is used, I'd assume that'd be the kind of thing you'd need to add as a warning for multiple releases before even doing something like having it switch to a more secure default on new install.
Criminally underused indeed. I have no idea why it's not more popular for 'average' users/orgs. I don't know what issues may come up with scaling this up, but in our small org we've been running 20-30 (mostly unprivileged) LXD containers in production for several years now for all sorts of intranet and external-facing services (auth, DB, web, etc). Sure, it requires a bit more thought to set up than Docker, but it's well-documented (for most people's uses at least), secure, stable and lightweight.
Maybe because many devs use Macs / Windows? Maybe WSL may tilt the balance in LXDs favour, but on OSX? Run it a VM yourself without the conveniences of docker-compose up?
As a Linuxuser myself, i looked at the competing solutions (podman, LXD, and the one by Canonical whose name i forgot) and thought: "Ain't gonna fly in a mixed environment."
They may be technologically superior, worse is better once again i guess. Would prefer them to Docker too.
I work with a few people who believed it was native, and used it for quite a while, until things started going wrong and we logged in to the vm to fix them.
And docker marketing has very effectively used that to their advantage.
On Linux you are going to run docker in VM anyway if you care a bit about security, but I know that almost all devs run docker directly on their laptops and with the user's full access to docker - ie. their user effectively becomes user. Without second thought ...
https://news.ycombinator.com/item?id=24782999 tongue in cheeck indeed, but better marketing it is, of course
Firewalld is implemented differently and will exhibit the expected blocking behavior: traffic targeting docker ports will encounter firewalld before it encounters docker.
This way you can make all your servers private and manage the firewall in a single access point to the outside world.
Making your servers public to the net by default and without a separate firewall solution is not so advisable in the first place.
I think software like docker have a responsibility to encourage secure-by-default configurations, but unfortunately "easy" is often the default that wins mindshare.
Either an exception should be raised or a default safe behaviour should be adopted when the example is encountered. I prefere breaking as soon as possible because the alternative is harder to debug.
I always see best practices like this, but they don't really help in grokking what's happening and why. I'd like to know more about the networking stuff, but whenever i look something up it's very specific, so you don't really learn why it's bad.
How can a regular user understand how the network stack works? At least enough to get an instinct why something would be bad.
https://aws.amazon.com/architecture/well-architected/?wa-len...
Take a look at the security pillar with extra care. For the cloud I would suggest you take a basic practitioner exam, or at least a preparation course in a platform like whizlabs. There you would get a basic understanding of how networking is laid on the cloud.
For private, on-premises projects, it really comes down to what you have at hand. In this case maybe the Google SRE book would be good. You take good practices in maintaining a data center and apply the distilled knowledge to what makes sense to your infrastructure:
https://landing.google.com/sre/sre-book/toc/index.html
Read this book as in topics, not sequentially, coming back to the fundamentals when you feel lost, otherwise you might end up lost in technicalities that make little sense to your work.
Also take a look at the shared responsibility principle. There it is exposed what are the client and cloud provider responsibility. When you have a private on-premises project all you have to do is implement the entire responsibility stack that the cloud does for you.
I don't like using systems that are complete black boxes, so whenever I use something, I try gain a reasonable understanding of how it works under the hood. If a system claims to make X easy, I want to know at least what is involved in accomplishing that, even if the implementation details aren't relevant knowledge. I don't often need to dig into the nitty-gritty of how the Linux TCP stack works, but even having a broad idea of how the TCP protocol works is pretty useful, and especially how it relates to other networking protocols.
I guess for practical networking, it helps to first focus on IP addressing and routing; ie. how does a packet sent from your computer actually get through all the switches and routers to the destination computer? The short answer is that every node (including your computer) makes a routing decision on where to send the packet, and then it's sent forward. This happens at each "hop" until it appears at the end (or gets dropped by a firewall).
And from this simple logic and some fancy tools to help you make dynamic routing decisions in response to changes in network topology (router went down? update local route information and send the packet to the other router that's still up), you can build the internet in a fault-tolerant manner.
So if you had 6379:6379 published anyone from the internet could connect to your Redis container. Oops.
> Always restrict your mapped ports to localhost (as in 127.0.0.1:3306:3306) unless you really want to expose it to the world
It's also worth pointing out that typically you don't even need to publish your database port. If you omit it, containers on the same network can still communicate with each other.
Very rarely do I end up publishing ports in production. If you wanted to connect to postgres you could still docker exec into the container on the box it's running on and interact with your db using psql that's already installed in that container.
A while back I wrote a blog post on the difference vs exposing and publishing ports at https://nickjanetakis.com/blog/docker-tip-59-difference-betw....
It should be trivial for me to lock down the ports of my instance, from the VPS web UI. AWS lets me create a new instance which is entirely closed to incoming connections except for port 22, which is closed except for whitelisting my IP address. This gives me good assurances even if my instance is running a vulnerable SSH server. It's also trivial to block outgoing connections, where that's appropriate.
It also means my instance is spared from constant probing, which keeps the logs clear.
DO has had a cloud firewall for a while now (a year or 2?) https://www.digitalocean.com/docs/networking/firewalls/. It's free too.
Looks like Linode has one coming soon https://www.linode.com/products/firewall/.
On top of that, you might also want to add `.git/` to your .dockerignore file, as it could significantly reduce the size of your image when calling `COPY`.
A more subtle issue I've noticed is the fact that `COPY` operations don't honour the user set when calling `USER`. The `COPY` command has its own `--chown` argument, which needs to be set if you'd like to override the default user (which is root or a root-enabled user in most cases).
I wrote up a similar article a while back, though it's focused on general best practices: https://lipanski.com/posts/dockerfile-ruby-best-practices
2. Given you do want system package updates, you need to deal with Docker caching breaking your security updates. So you need to rebuild your image from scratch, without caching, either every time there is a security update or on a weekly basis. https://pythonspeed.com/articles/docker-cache-insecure-image...
3. Not mentioned: don't use build secrets via ARG.
Some approaches to secure build secrets:
1. BuildKit supports them: https://docs.docker.com/develop/develop-images/build_enhance...
2. Via the network, which is a hack, but it works: https://pythonspeed.com/articles/docker-build-secrets/
3. Via multi-stage builds, but this destroys caching.
Generally, if you want to update system packages, rebuild the container make it the new base for you. Updating with every build provides a potentially non-reproducible build.
> If you rely on latest you might silently inherit updated packages that in the best worst case might impact your application reliability, in the worst worst case might introduce a vulnerability.
On this one I disagree. Your CI should handle reliability (and if not, you have a bigger problem) and you're more likely to patch a vulnerability than to introduce a new one and it's unlikely that by the time a PR hits production that the version is compromised.
I understand that updating cuts both ways when it comes to security, but I agree with Matt Tait's ultimate conclusion when he spoke on this issue a few years ago: For most medium size and smaller companies constantly updating is safer than delaying. He had real world data and graphs of compromise windows, etc. Short answer was that attackers are more time motivated than defenders.
There are a lot of semi-official Docker images on Docker Hub that are published once and never updated until the next software release. That is a huge anti-pattern in my view, those images should not be relied on for production.
For the projects I've been on, the latest version plus the test suite is enough to catch weirdness creeping in and, as a side benefit, it gets fixed faster than if it were pinned. Sometimes the issue really is caused by the base image and it is easier to get a fix merged if the issue was caused quite recently because the developers responsible see early reports of issues as more endemic than if they're reported days or weeks later.
Eg. postgres:12 instead of either postgres:latest or postgres:12.0.1
I've always tried to pick stable tags [1] if they're available.
1. https://docs.microsoft.com/en-us/archive/blogs/stevelasker/d...
> It simply happens to be the default tag when omitted
That makes it special though, right? It's different than any other tag because it can be inadvertently pushed via accidental omission.
https://github.com/safe-waters/docker-lock
Would love for anyone dealing with this issue to check it out!
I would be fine taking a healthy performance hit if I knew that the base OS was secure. (At this point I expect the BSD folks to chime in that they have had this for years)
On the other hand, changing the docker user to non root might introduce some failure scenarios (eg file ownership) which might lead to other problems like availability incidents.
Security should start with threat modelling, and taking a risk-based approach. You can spend hours fearing that someone might break out of the Docker virtual machine through a zero day, instead of using that time to fix much more likely and plausible threat scenarios. Pick your battles.
If you application needs root to execute, with very few exceptions, it is already wrong.
If you're in that boat, there isn't much you can do except work with it.
That's the case I had in mind when writing that quote.
That's not the case, either. And root inside the container != root outside the container. A completely new user:group namespace is created inside the container. This is, in very large part, what Linux namespaces are for.
Further, you can certainly have a root-owned file accessible to non-root users, via chmod bits.
There are only a handful of excuses, ever, to run a privileged container. If you're not 100% sure, then it is not one of those excuses.
No. root inside is root outside (if you can get outside). The behavior you describe only applies if you enable user namespace remapping, which docker doesn’t by default.
edit: I joke, I love what containers accomplish, and working with Docker has been a joy (:
As a result, the test needs to be run using sudo to work correctly. The test environment is containerized, so there is a Dockerfile, but as the article mentioned the user in the container should not be root. Is there any plausible way around this? Updating the image is not under my control, and changing the copy code is also riskier than I would like.
Since this is in the context of CI, the threat is lower than live production, but still....
Two ways this was useful for us. Firstly, we needed a private key in the image to pull some private git repos. By doing this in a previous stage, they're not included in the final image layers. Secondly, we have a python backend and small react app served from the same image. By splitting their build steps into a backend and frontend stage, changes to frontend code don't break caching for the later backend steps or vice versa. E.g.
FROM python:3.8.3-slim-buster AS frontend
# Do frontend build steps
FROM python:3.8.3-slim-buster AS backend
# Do backend build steps
FROM python:3.8.3-slim-buster AS final
COPY --from=frontend /app /app/frontend
COPY --from=backend /app /app/backendReading this thread I'm surprised this isn't common knowledge by now given how it's so incredibly paramount to efficient production releases.
This sounds wrong. If secrets are in the environment they are not in the Dockerfile, so they are NOT distributed with the application.
Injecting environment variables at runtime, however (through docker run -e or whatever orchestration system you're using), is good.
It was the heading that got me on the wrong path, I think that should be clarified further:
> Do not store secrets in environment variables
Because PID 1 has that env, all processes spawned from that can read all of those.
I prefer mounting them to /run/secrets via tmpfs. Which can also have selinux policy attached.
This way, someone else cannot read them by spawning shell inside container
Seems such an obvious tip first up that it put me off reading the rest of the article.
It should be totally OK to upgrade the packages.
Also I don't get that it's advised to use "apt update" when you can't do "apt upgrade". What's the point??
`rm -rf /var/lib/apt/lists/*` in the same RUN decreases bloat and I think possibly decreases cache misses as well.
Unlikely, you'd still have misses because of stuff like file metadata mismatches (think modification times).
But yeah, that last part doesn't make any sense. If you're not running `apt-get upgrade`, it doesn't make sense to run `apt-get update` as nothing is using the newly fetched data anyways...
If there's a security issue you need to rebuild your images anyway and optimally you have a system in place that represents images and their dependencies as some sort of dependency graph structure, so when you upgrade your base image all dependent images get rebuilt automatically.
https://github.com/bazelbuild/rules_docker
rules_docker allows you to create byte-for-byte reproducible container images on your system, without even having a Docker daemon installed. So much cleaner to use than 'docker build' once you get the hang of it!
USER somenonrootuser
My current strategy is to let the container entrypoint start as root, but then I have a wrapper program installed that drops privileges before it exec()s the actual service. It works, but is there a better / more accepted way of doing this?
If you don't control the services, maybe in the Dockerfile you could write a dummy config file and chmod it?
..If you don’t inspect the wget script you might as while pipe it into bash.
.. How to distribute secrets if not by env? (which I agree! Honest question)
There are competing pieces of advice for secure distribution of secrets, but my current preference comes down to one of these ways, depending on the organization:
1. OpenShift/Kubernetes Secrets mounted into the Pod at runtime.
2. Hashicorp Vault (has a really well designed API. It's very usable just with curl, which makes using it a joy)
3. Sealed Secrets (less experience here but it's looking positive right now) - https://github.com/bitnami-labs/sealed-secrets
If you're using a different PaaS besides OpenShift, it may also offer options worth considering (although do think about portability. These days apps move platforms every few years on average, though I think that may be changing now that K8s is becoming the standard).
Do you recommend mounting secrets as environment variables to the kubernetes pods instead of files?
You'll want to use BuildKit (`docker buildx`), see https://docs.docker.com/develop/develop-images/build_enhance...
[edit] My bad, that works for secrets needed at build time, not at runtime of course.
If you need to provide a secret value to an image and it needs to remain secret (like a database password), you most commonly would set the env values at runtime or volume mount a config file at runtime.
On a different side of this, if you need a secret at image build time (like an SSH key to access a private repo), you can use build arguments with the ARG keyword and they won't persist into the final image. Multi stage dockerfiles are also a great way to keep your final image lean and clean.
- A vault (Conjur, HCV, something else)
- A built-in credential service that comes with your cloud
- A sidecar that injects credentials or authenticates connections to the backend directly (Secretless Broker, service meshes, etc)
If you are doing a poor man's solution, mounted tmpfs volumes that contain secrets are not terrible (but they're not really that much safer than env vars).
common info pages (ex: phpinfo), core dumps, debug errors and logs are notorious for containing them. And those aren't even counting the ways a malicious actor can persuade a program to provide them.
I've found its best to use the secrets provider that comes with your cloud provider.
For AWS using SSM's get_parameter seems the best thing. But it means you need to find a custom shim to put in your container that will go and fetch the secrets and put them somewhere they are needed.
There is a Google Cloud KMS keyring (for typical usage) and a GPG key (for emergency/offline usage) set up to handle the encryption/decryption of files that store secrets for each application's deployment. I have some bash scripts that run on CI which are essentially just glorified wrappers to `sops` CLI to generate the appropriate `.env` file for the application, which is put into the container by the `Dockerfile`.
Applications are already configured to read configuration/secrets from a `.env` file (or YAML/JSON, depending on context), so this works pretty easily and avoids depending on secrets being set in the `ENV` at build time.
You can also, of course, pass any decrypted values from `sops` as arguments to your container deployment tool of choice (e.g. `helm deploy foo --set myapp.db.username=${decrypted_value_from_sops}`) and not bundle any secrets at build time at all.
Unfortunately installing seems non-trivial. Would love simple binary or .deb/.rpm install.
Nevermind Dockerfiles, env variables are preserved in stopped containers that are hanging around even when you use docker-machine AFAIK, you can easily `docker inspect` them.
I've helped a lot of people with Dockerfiles ranging from horrendous security issues to simple bad practice making lives hell, and much of this is solid advice.
A lot of what I tell people boils down to: keep your container lean and clean. Don't do things in the container that you wouldn't do on the host (like curl-ing from the internet into bash as root :-D, or using questionable base images).
My deployment life has been vastly improved by shipping in containers, but I have seen a lot of security regressions because people feel safe to be reckless (like running the app as root) due to the container guard rails. Don't think this way.
> Do not upgrade your system packages
Most distros will have smooth upgrades and provide you with patched libs that your app may need and the latest image may not provide. It's slightly more prone to breaks but it creates a less vulnerable runtime app env.
> Do not use ‘latest’ tag for base image
Depends on the org but sometimes pinning means that you will likely end up using and end-of-life image because it requires proactive work to maintain. If you leave it as 'latest' this won't happen but you will get out-of-band breaks to keep that working. Choose wisely.
A few things I would add too:
- Don't mount Docker socket into any container unless absolutely necessary
- Your biggest security threat will be from your app's dependencies, not the container's setup
- Do not run a full init system unless absolutely necessary as this is just a security disaster waiting to happen. There are valid use cases for it but they're rare.
Many must run as root, and the reasons not to do that are discussed in the article this HN thread is discussing.
Systemd is particularly tricky because it needs to be able to control the cgroups of its child processes, which means the container needs to be granted that capability. See https://developers.redhat.com/blog/2019/04/24/how-to-run-sys... about how to run systemd in a container via Podman, and is a follow up to https://developers.redhat.com/blog/2016/09/13/running-system... which discusses why the Docker case is even more difficult.
That said, if you just want a process supervisor for a multi process container, there are several more minimal init systems that will work well, for example, supervisord.
For security using the latest versions of both base images and packages is typically a good thing. The cases that the newest package is more vulnerable since something 1, 2, 3 years old are not that common.
However, if your process requires reproducibility/traceability (medical and other regulated domains) you cannot just deploy the latest and greatest. You need to pass it through some release process first. That should not be an excuse to run outdated, vulnerable software though. The same holds if you require high availability. Even if you might not need to document what you are using, you want to test whether it causes performance issues (zero performance is the worst one...).
the ubuntu:20.10 now want to upgrade the "libssl1.1"!
docker run --rm -it ubuntu:20.10 bash -c "apt update && apt upgrade"
...
The following packages will be upgraded:
debianutils diffutils findutils gcc-10-base
libgcc-s1 libgnutls30 libprocps8 libssl1.1 libstdc++6
libsystemd0 libudev1 procps sed zlib1g
docker run --rm -it ubuntu:20.04 bash -c "apt update && apt upgrade"
...
The following packages will be upgraded:
gcc-10-base libgcc-s1 libstdc++6 zlib1g20.10 is not a LTS, so it's more likely to get semi-spurious updates than a LTS version like 20.04 or, say, debian stable.
For most of my non-alpine-based images, I use debian:buster-slim as base, as it's got a fairly stable base and gets quite routinely updated:
$ docker run --rm -it debian:buster-slim bash -c "apt update >dev/null 2>&1 && apt upgrade"
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.this image has been upgraded "39 hours ago".. so you have to check 1-2 month later
debian buster-slim f49666103347 39 hours ago 69.2MBI check for base image updates every day, and it's one of the most oft-updated ones -- hence my preference for using it as "the" base of all others.
To give a specific example, on Kubernetes you store your Private Key and Certificate as a secret (there’s a special “type” of secret for this, but K8s doesn’t actually treat it any differently). You then mount that certificate as a volume (in /var/run/secrets/tls, or wherever you define it), and it shows up as a normal file, accessible to whichever user your container runs as.
So for clarity then, if somebody shells into the container - at that point it's still sitting there mounted and they can read the file? Or does k8s somehow manage this in a way that only the web server process can see it? Or do we just accept that you have to treat anyone with access to kubectl as authorised to know your private cert ?
Yes, if someone has exec access into your container, they'll be able to see the secret, unless you do something like making the K8s secret an already encrypted blob, and then in process decrypting it again and reading it. If someone's got exec access to your container though, you've got bigger problems.
> Or do we just accept that you have to treat anyone with access to kubectl as authorised to know your private cert ?
You can set up access so that someone can login with Kubectl but still only be readonly, and you can preclude read ("get") access to secrets to the readonly role, so they won't be able to view the contents of the secret.
Adding the certificates into the image... don't do that :)
A perfect example of this would be passing your NPM_TOKEN to install company scope packages.
What's the best way to handle this?
Ah, the joys of working somewhere that isn't required to document, answer for, and ultimately remediate every CVE that is present in any package installed on any of your containers within your production application. Sadly, compliance and regulatory oversight don't leave this option open to everyone.
There's good reasoning against the concept of barebones containers, but unfortunately everything from bricks, knives, and well-reasoned arguments all bounce harmlessly off of regulations and external compliance requirements.
Yes, definitely don't put secrets in the Dockerfile itself. I'm curious if there are reasons not to use a .env file though?
> Only use trusted base images
This is a good sentiment, but docker hub also has plenty of images that are built directly from a github repo. You can inspect the Dockerfile and (as long as you trust Docker) trust that it was built as written. In this case I recommend pinning to a specific container SHA (image_name@sha256:...) in case the source repo gets compromised. For official images, you can pin to a tag IMO.
> Do not use ‘latest’ tag for base image
Regardless of the security concerns, pinning to latest will probably bite you when there's a major version bump (or even maybe a minor one). Imagine you built a container on ubuntu:latest when latest was 20.04 and some new employee gets hit with 22.04. That's a bad surprise.
> Avoid curl bashing
Evergreen fight here, but I agree with the author that you should at least validate checksums on external files before executing or using them.
> Do not upgrade your system packages
This is where the throne of lies about Docker idempotency crumbles. You have to apt-get update because Debian/Ubuntu repositories remove old versions when there are security fixes (not 100% sure on this, feel free to correct me). So if the ubuntu:20.04 image is released and there's a security update in openssh-client, running "apt-get install openssh-client" without "apt-get update" will fail. So we all run "apt-get update" and pretend that the containers we build are time-invariant. They're not, and in fact we occasionally get security updates snuck in there. Luckily Debian and Ubuntu do a good job not breaking things and no one complains. But if you build a container on Tuesday it's not guaranteed to be the same as on Wednesday and there's nothing you can do about that with a Debian distro. But it's actually fine in practice so we pretend not to notice. The point about not running "apt-get upgrade" is kind of moot - you're going to effectively be upgrading whatever you "apt-get install", so it's probably worth taking the same trade on the built-in packages.
Importantly, there's no security risk - just a configuration one.
> Do not use ADD if possible
Same as above - you should be checksumming external dependencies or self-hosting them.
> Do not root
> Do not sudo
I haven't thought deeply about these. I'd prefer to trust the container system isolation rather than playing with users and permissions. I don't understand this risk well enough to have a well-formed opinion.
Because often that .env file will get checked into source control (sometimes intentionally, sometimes accidentally), and then you have secrets in your git history that you need to rotate.
In general the best thing to do is store your secrets in a place/service specifically designed for secrets, and either fetch them directly into the container in your entrypoint, or pull them into a location that you mount as a volume in the container.
> I'd prefer to trust the container system isolation rather than playing with users and permissions.
Don't. Containers don't give you full isolation. The 'root' user inside the container is the same 'root' user as outside, and container escapes may be possible. A good defense-in-depth strategy suggests that you should run things using the least privileges possible, and that doesn't change just because something is running in a container.
I would do that by getting the checksum off the website, and hardcoding it into my Dockerfile, right? At that point I've pegged to a version, and I might as well just keep the binary local and COPY it in.
Is that right? And am I accidentally reviving the 'evergreen fight'? :)
> Yes, definitely don't put secrets in the Dockerfile itself. I'm curious if there are reasons not to use a .env file though?
Environment variables are a terrible place to store secrets, regardless of whether you're using docker.
Environment variable values get dumped all over the place. /proc/*/environ, docker inspect, /var/log/..., core dumps, error messages, info pages (phpinfo), etc. Also, unlike file handles and secrets services (hashicorp vault, etc), every child process inherits all of its parents' environment variables, greatly increasing the attack surface.