I think it's counterintuitive, but I learned the hard way that 3306:3306 will automatically add a rule to open the firewall on linux and make MySQL publicly accessible.
So if you had 6379:6379 published anyone from the internet could connect to your Redis container. Oops.
> Always restrict your mapped ports to localhost (as in 127.0.0.1:3306:3306) unless you really want to expose it to the world
It's also worth pointing out that typically you don't even need to publish your database port. If you omit it, containers on the same network can still communicate with each other.
Very rarely do I end up publishing ports in production. If you wanted to connect to postgres you could still docker exec into the container on the box it's running on and interact with your db using psql that's already installed in that container.
A while back I wrote a blog post on the difference vs exposing and publishing ports at https://nickjanetakis.com/blog/docker-tip-59-difference-betw....
It should be trivial for me to lock down the ports of my instance, from the VPS web UI. AWS lets me create a new instance which is entirely closed to incoming connections except for port 22, which is closed except for whitelisting my IP address. This gives me good assurances even if my instance is running a vulnerable SSH server. It's also trivial to block outgoing connections, where that's appropriate.
It also means my instance is spared from constant probing, which keeps the logs clear.
DO has had a cloud firewall for a while now (a year or 2?) https://www.digitalocean.com/docs/networking/firewalls/. It's free too.
Looks like Linode has one coming soon https://www.linode.com/products/firewall/.
There's no way this is true. This completely defeats the purpose of a firewall.
If it is happening, then it's Docker doing this - not the linux firewall that comes stock with most distribution (iptables and the like). They would never simply add rules to the chains just because something was listening on that port.
Really, the best security advice for using Docker is to not use Docker. Unfortunately, there aren't very many "hold your hand" alternatives available. Aside from LXD and that family of technologies, which are criminally underused.
It's not so much that they explicitly open a firewall rule, as that they take a networking path that isn't really covered by traditional firewalls.
Another way of viewing it is that Docker "is" your firewall for your container workloads, and that adding a port-forward is equivalent to adding a firewall rule. Of course, that doesn't change that public-by-default is a bad default.
Terrible practice, in my opinion. Docker shouldn't be touching firewall stuff.
For most container purposes, host networking and the default process namespace is absolutely fine, and reduces a lot of problems with interacting with containerized apps. 95% of the use case of containers is effectively just a chroot wrapper. If you need more features, this should be optional. This would also make rootless federated containerized apps just work. But nobody wants to go back to incremental features if Docker gives them everything at once.
It's not as powerful as the later tables in the chain (see https://upload.wikimedia.org/wikipedia/commons/3/37/Netfilte... ) but a lot more robust.
My current policy is to set `"iptables": false` in Docker's `daemon.json` on any public machine. I don't understand why this isn't the default.
If you don't muck with iptables then you need a (slow) userspace proxy to expose your pods. That also means losing things like the source IP address for any incoming connections.
I do see that RemoteAddr is from a private IP range. Luckily I'm not using this information anywhere, but good to know.
It’s easy to blame Docker but I’ve seen this failure mode happen many times over the years - even experienced admins make mistakes. As always, defense in depth and continuous monitoring trumps raging about software.
The default is terrible. It got into 1.0 and just got stuck there :(
Changing defaults on such widely used software is, unfortunately, hard.
For whatever reason Docker prepends rules. There are things you can do to add your own filtering (Docker forwards to the "DOCKER-USER" chain where you can put your rules), but it requires people to know what is happening I order to use it securely.
It would be really nice to be in a more secure situation by default... open to suggestions and contributions for Docker 21.
The nice thing is admins can already define a default value that the eninge will use to bind to (when no address is specified on -p). Warning can point users to that setting.
This also raises the possibility of different security profiles like dev, prod, etc.
A default Docker install would be documented as being for development, and you run "docker secure" to change that for other environments.
Is there official documentation that tells users to set the default bind address as a best practice?
I wasn't thinking so much of just changing one setting, but rather having a way to easily reconfigure an installation to set multiple settings to improve security.
In addition, elevating this to the status of a command and documenting it as a best practice helps spread awareness.
I like Godel_unicode's suggestion of logging and that could probably done in a stronger manner if there was some point (post-install, maybe starting a container) where it checked the existing rules and used a more prominent warning when there are existing rules which would prevent a container which would be reachable now from being reachable in the future. Given how widely Docker is used, I'd assume that'd be the kind of thing you'd need to add as a warning for multiple releases before even doing something like having it switch to a more secure default on new install.
Criminally underused indeed. I have no idea why it's not more popular for 'average' users/orgs. I don't know what issues may come up with scaling this up, but in our small org we've been running 20-30 (mostly unprivileged) LXD containers in production for several years now for all sorts of intranet and external-facing services (auth, DB, web, etc). Sure, it requires a bit more thought to set up than Docker, but it's well-documented (for most people's uses at least), secure, stable and lightweight.
Maybe because many devs use Macs / Windows? Maybe WSL may tilt the balance in LXDs favour, but on OSX? Run it a VM yourself without the conveniences of docker-compose up?
As a Linuxuser myself, i looked at the competing solutions (podman, LXD, and the one by Canonical whose name i forgot) and thought: "Ain't gonna fly in a mixed environment."
They may be technologically superior, worse is better once again i guess. Would prefer them to Docker too.
I work with a few people who believed it was native, and used it for quite a while, until things started going wrong and we logged in to the vm to fix them.
And docker marketing has very effectively used that to their advantage.
On Linux you are going to run docker in VM anyway if you care a bit about security, but I know that almost all devs run docker directly on their laptops and with the user's full access to docker - ie. their user effectively becomes user. Without second thought ...
https://news.ycombinator.com/item?id=24782999 tongue in cheeck indeed, but better marketing it is, of course
Firewalld is implemented differently and will exhibit the expected blocking behavior: traffic targeting docker ports will encounter firewalld before it encounters docker.
This way you can make all your servers private and manage the firewall in a single access point to the outside world.
Making your servers public to the net by default and without a separate firewall solution is not so advisable in the first place.
I think software like docker have a responsibility to encourage secure-by-default configurations, but unfortunately "easy" is often the default that wins mindshare.
Either an exception should be raised or a default safe behaviour should be adopted when the example is encountered. I prefere breaking as soon as possible because the alternative is harder to debug.
I always see best practices like this, but they don't really help in grokking what's happening and why. I'd like to know more about the networking stuff, but whenever i look something up it's very specific, so you don't really learn why it's bad.
How can a regular user understand how the network stack works? At least enough to get an instinct why something would be bad.
https://aws.amazon.com/architecture/well-architected/?wa-len...
Take a look at the security pillar with extra care. For the cloud I would suggest you take a basic practitioner exam, or at least a preparation course in a platform like whizlabs. There you would get a basic understanding of how networking is laid on the cloud.
For private, on-premises projects, it really comes down to what you have at hand. In this case maybe the Google SRE book would be good. You take good practices in maintaining a data center and apply the distilled knowledge to what makes sense to your infrastructure:
https://landing.google.com/sre/sre-book/toc/index.html
Read this book as in topics, not sequentially, coming back to the fundamentals when you feel lost, otherwise you might end up lost in technicalities that make little sense to your work.
Also take a look at the shared responsibility principle. There it is exposed what are the client and cloud provider responsibility. When you have a private on-premises project all you have to do is implement the entire responsibility stack that the cloud does for you.
I don't like using systems that are complete black boxes, so whenever I use something, I try gain a reasonable understanding of how it works under the hood. If a system claims to make X easy, I want to know at least what is involved in accomplishing that, even if the implementation details aren't relevant knowledge. I don't often need to dig into the nitty-gritty of how the Linux TCP stack works, but even having a broad idea of how the TCP protocol works is pretty useful, and especially how it relates to other networking protocols.
I guess for practical networking, it helps to first focus on IP addressing and routing; ie. how does a packet sent from your computer actually get through all the switches and routers to the destination computer? The short answer is that every node (including your computer) makes a routing decision on where to send the packet, and then it's sent forward. This happens at each "hop" until it appears at the end (or gets dropped by a firewall).
And from this simple logic and some fancy tools to help you make dynamic routing decisions in response to changes in network topology (router went down? update local route information and send the packet to the other router that's still up), you can build the internet in a fault-tolerant manner.
Is that true? 3306:3306 would bind the port on all interfaces, but I was under the assumption you'd have to explicitly enable firewall port 3306 for the machine to accept traffic to port 3306 from outside of your machine.
I'll have to test that.
When an IP packet related to your container arrives (<host ip>:<host port>):
- docker rewrites it to target <your container's ip address>:<your container's port> (NAT table, chain PREROUTING delegates to DOCKER chain with the relevant DNAT entry)
- since the IP does not match your host, the packet is forwarded (there's a relevant routing entry pointing to docker's virtual interface)
- the first thing you encounter in the FORWARD chain of the FILTER table is a few jumps to the docker-related chains, DOCKER chain in particular accepts all packets destined to <your container's ip address>:<your container's port>
So a few takeaways:
- your standard firewall might not be involved because its chains are plugged in after the docker chains in the FORWARD chain (e.g. ufw under Ubuntu)
- if the above is true and you want your firewall to matter, you have to add stuff to DOCKER-USER chain in the FILTER table
- at that point the host port and IP doesn't matter since it's already been mapped in the NAT table's PREROUTE chain at the beginning of processing - write your firewall rules to address specific containers