Who here is willing to name a few downsides of using containers?
* increased complexity if you have a simple setup; for example if your deployment now consists of copying just 1 file or 1 folder and starting up an app/service, then using containers will include additional moving parts in your setup
* if you already have legacy tech, migrating won't be straighforward, especially for stateful applications (think JSF, ASP.NET Forms, etc.)
* as jeremyjh pointed out below, unless you have a specific strategy for this, you will have duplicated dependencies in containers, which need to be checked for vulnerabilities; you can't just apt update && apt upgrade on the host system from just the security repositories and call it a day (more or less)
* an extra layer that comes with its own bugs
Maybe I need to enable it in containers too? I have no idea how to use it on (containerized) Alpine Linux though.
Even without unattended upgrades, finding the list of `Dockerfile`s and `docker-compose.xml`s that might be affected by a new vulnerability sounds more complicated to me. Until now it hasn't been that difficult but I'm still a bit nervous if I somehow missed some vulnerable images.
That's not a container specific problem. That happens even when you don't use containers.
Making maintenance and updates harder is definitely a container specific problem.
Over-simplicity becoming complexity. You used to be able to write an app in C, pre-build it for a dozen platforms, then run it on a dozen platforms. Now you have to run everything on one platform (a virtualized containerized Linux environment) because anything else is literally impossible. You can still run the app anywhere - it just has five or six layers of virtualization, abstraction and tooling now.
More complexity. Containers are supposed to just be processes, but you can't run them without preparing the host OS first, and you can't really use them en-masse without a complex scheduler and tooling.
Nobody mentions it, but be careful your disks don't fill up just from deploying regularly. People think containers "cost almost nothing", and then all of their over-use of containers results in resource drain from the 15 million things necessary to configure and build and deploy and manage the things across distributed clusters. It's more resource-intense than people advertise.
The container is supposed to be this read-only isolated self-sufficient jail, so good luck getting anything out of it without a bunch of hoops. You seriously need to specify 50 different config values just to do simple things like networking, writing to a persistent disk, picking what to run or stop, interacting with other apps.
People say "a container is just another process" - except each one may be using an entirely different Linux distribution worth of software dependencies. And you still have to track what containers you're running, because you will depend on X release of Y container.
Functionally speaking, containers are almost like a VM inside a VM. It gets that obtuse.
I don't know that devs are actually saving time using containers vs a single dev platform. I think the problem all along was nobody knows how to maintain platforms, so they literally create and throw away whole platforms every time they build an app.
Most of these comments are about Docker, but not necessarily specific to it in practice.
Apparently it is not only unnecessary, it is an anti-pattern: https://www.socallinuxexpo.org/scale/15x/presentations/confi...
> You used to be able to write an app in C, pre-build it for a dozen platforms, then run it on a dozen platforms. Now you have to run everything on one platform (a virtualized containerized Linux environment) because anything else is literally impossible. You can still run the app anywhere - it just has five or six layers of virtualization, abstraction and tooling now.
I see more and more "here is a Docker image!" things pop up for tiny shell script utilities on github. It's like autotools with 100x the complexity and "portable" to exactly one platform. People have no clue what they are doing, they just want any excuse to use Docker.
> The container is supposed to be this read-only isolated self-sufficient jail, so good luck getting anything out of it without a bunch of hoops. You seriously need to specify 50 different config values just to do simple things like networking, writing to a persistent disk, picking what to run or stop, interacting with other apps.
This is my biggest pain point with containers, and where I feel Docker marketing is lying the most. Docker is pitched as making development easier, when in fact containers significantly impede debugging. You even need special tools (like https://github.com/emacs-pe/docker-tramp.el) to do the most basic tasks like look at files. The marketing hype is that all you need to solve production problems is a log server and a few line graphs of metrics (better hope you instrument your app code for the right metrics...). Add in microservices and now you need distributed tracing too. This extra infrastructure is a ton of work to set up for small apps.
If it's immutable containers we're talking about, then patching needs to be handled by your image build system and containers need to be restarted in order for the patches to be applied. Yes, you'll need an image build system and at least one staging environment.
Applications need to be designed with immutable containers in mind. This makes them a pretty niche technology.
LXD, which offers mutable containers, doesn't suffer from those last two points. Of course you don't get the benefits that the immutable design brings.
Microsoft are obviously investing quite a lot of effort in making the windows container ecosystem more robust, as well as improving the experience of running Linux containers on Windows.
afaik windows containers only support a subset of the full win32 api, so it's exactly the same as running actual windows. that said, simple asp.net sites should work fine
However, layers of abstraction should also bring benefits wherever they're used. Docker and Kubernetes go hand-in-hand with distributed software or "microservices." If you don't see any benefit to that architecture, you're likely going to introduce unnecessary complexity into your infrastructure with containers.
The more deployments of individual software applications you make, the more you can gain from containers. And the inverse is also true - if you rarely deploy new versions or updates to your software, using containers will require tackling complexity that doesn't benefit your use-case.
When you launch processes on Linux, you can choose for all the processes to share memory with each other (in which case we call them threads), or the processes can have separate memory. We call this capability "virtual memory", and it allows processes to pretend that they and the kernel are the only software running on the physical machine. (For example, every process might think that it can write to the memory address 0x08048000, but in fact that address is virtualized and mapped by the kernel to some different physical memory address.)
Though we take it for granted, this memory isolation between processes was the first form of kernel namespace. Modern Linux now has many of these namespaces, which can provide a virtualized view of the file system, network, IPC, uids, and so on. "Containers", then, are processes that employ many of these namespaces to provide isolation from others on the system. (To extend the analogy: just like regular processes think they have their own memory, containerized processes can have their own file system with a distinct /tmp, distinct process id namespace, or namespace of users that exist on the system. Every process can run as PID=1 if it wants, within its own PID namespace.)
These namespace primitives, from which containers are made, are present on modern Linux systems whether one chooses to employ them or not. The primary downsides to consider are efficiency of the system, and the complexity of managing it. If your system (CPU & OS kernel) implements virtual memory efficiently, then there isn’t much downside to using it. The same is probably true for most kernel name spaces, though file system and networking virtualization can have a performance hit if not configured with care.
Kernel namespaces are a core feature of Linux and can be taken advantage of without using technologies that are specifically "container-oriented". For example, consider systemd: it provides a comprehensive set of parameters for employing namespace features in processes that it launches. Parameters like PrivateTmp=yes or PrivateDevices= [1] will virtualize just one part of the system (/tmp or /dev and such, respectively). PrivateUsers= and PrivateNetwork= allow you to launch with a new user or network namespace. It's really that easy to employ namespaces: add a line to your application's unit file.
When you look at container technology as a set of system primitives, then it makes sense to consider employing them individually. They don't have to be used in an all-or-nothing way. You might give your daemon access to the regular network with a private file system, or you might use the regular file system too while virtualizing just /tmp and /dev. You can isolate processes in whatever way makes sense, and share whatever resources make sense. If a process doesn't need access to the network, and doesn't need a shared /tmp with the rest of the system, then there's little downside to giving the process its own namespace. Indeed, I expect it will increasingly be considered a best practice for services to isolate themselves according to the principle of least privilege. This reduces the chance that components will interact with each other in unexpected and undesired ways, and may have some security benefits.
(If the question is about using Docker or some other specific technology as part of a development and/or deployment process, then that's a different topic. Specific technology suites have tradeoffs beyond those of kernel namespaces.)
[1] https://www.freedesktop.org/software/systemd/man/systemd.exe...
This shouldn't be underestimated, though!
I agree that efficiency is a no-brainer for almost all purposes, especially compared to other forms of virtualization like KVM.
But: Managing the complexity can be tough, it sometimes feels like creating all low-level firewall rules by hand for a large network. Well, for firewalls you have stuff like firewall generators, in the context of containers these are projects like Docker or LXD. But they share the same downsides: You may have trouble to see what happens behind the scenes.
The largest risk here, however, is not the working time spent to learn and apply that stuff. It is a false sense of security where your complex system might, in the end, to have holes which you assumed should not exist.
For example, not many people are aware that if the user can start docker instances, they are effectively root.
(One might argue that this is more a failure of docker than of containers. But then, why do people use complex monsters like docker in the first place? Because managing plain containers is a pain and other complex building around containers have their downsides, too.)
[1]: Run within Kubernetes: https://github.com/kubernetes-incubator/cri-o [2]: Run Standalone: https://github.com/projectatomic/libpod/blob/master/docs/pod... [3]: Build: https://www.projectatomic.io/blog/2017/11/getting-started-wi... [4]: Move: https://rhelblog.redhat.com/2017/05/11/skopeo-copy-to-the-re...
Process memory has traditionally been an "implementation detail" of the process. It typically doesn't need to be shared, the addresses do not need to be fixed, etc. (Shared memory is an exception of course).
Resource like the file system or network, on the other hand are mostly shared. It does matter to the outside world through which port the program is reachable, it matters where it puts its files, etc. Again, there are exceptions of course, like temporary files.
Now the question is how can we pull out the configuration, like "what port must the program listen on?", and decide these tasks centrally. The container solution is (if I understand correctly) that the program will always listen on port 80, but the container configuration diverts that port to be accessible to the outside world as 8080, etc.
I predict that this is going to be a failure in the same sense that OOP has been a failure (if you agree with me on that point). It becomes a tangled mess of layers and indirections, and incredibly hard to debug.
Instead I think it is much cleaner to make the configurations like, "What port should I listen on?", "Where can I put my temporary files", easily accessible to the program. The program should not be required to define its own configuration for all the problems which containers want to deal with today. Instead it should import this configuration from a central location through a well-known interface.
There must be a simple convention where these decisions can be configured. The configuration must be easily accessible to A) the system, B) the program, C) the administrator. It's been customary to put configurations in text files for a long time, and that's mostly good. However what's been missing is that most programs have had their own configuration file written in a custom syntax and stored in a location that are not easily known by the rest of the system. The system is not really aware that the program is configured e.g. to listen on port 80. And the administrator needs to learn new configuration files for every program.
I'm not proposing a central registry where all the cruft of the system should accumulate. Instead, just for the concepts that are shared by many programs -- the resources which containers virtualize today -- should be configured through a system-wide API.