Running containers without Docker
jvns.ca
jvns.ca
There are lots of great tools in the Docker/container ecosystem these days, but I can see the argument that a major infrastructure migration should be done in incremental steps. Using containers as simply a Puppet replacement strikes me as a really good idea! You want your application to be _ready_ for containerization, before you try to introduce an orchestration system.
Although, when she mentioned that her team is meant to be "Heroku for the rest of the company", I couldn't help but think of Convox [0]. It's a great tool, it's open-source, and using it is really just like having your own Heroku. We used Convox at my last company, had amazing success with it, and I've been recommending it ever since.
[0] https://jvns.ca/blog/2016/10/26/running-container-without-do...
It's a simple of layer around AWS so it doesn't add extra craziness some other approaches do.
As for the OP, her approach is spot on!
Containerizing your app is the first step in future proofing it. Doing so correctly will remove layers and therefore simplify things.
You now are looking at a package that can run on a laptop and run on a server with traditional techniques (upstart, systemd, etc).
You may take this to the next step and enter the orchestration world. Or not! Patience is a virtue, wait 6 more months and all the orchestration tools will be that much better.
It's all about simplifying. Adopt Dockerfile and shed Homebrew and Chef. Run containers on a "dumb" box with systemd and shed Packer.
Run containers on ECS and shed custom AMIs and userdata and nagios and...
Presumably ECS has its own alerting mechanisms similar to CloudFormation and ElasticBeanstalk (I've only used the latter two, not the former).
Maybe ECS is different. I haven't played with it, but I imagine it's "make an SNS topic" like everything else.
For example Docker was based on LXC till 0.9 but was so successful in hype and misdirection of the project it's based on that till today most commentators here do not seem to have a proper idea of the LXC project and whatever ideas they have are negative. How did this come to be?
I don't know how healthy it is for open source if the HN community lets VC funded projects essentially hijack open source projects in this way and allow misconceptions to grow about them. For the record LXC has always been a full scale container manager and was and remains a much simpler way to use containers than Docker.
Most of the big problems holding back containers like security, isolation, multi-tenancy are kernel side and not container manager side or ecosystem side and will be solved kernel side. Similarly most of the features of containers that are usable today are also thanks to kernel side.
Yet the folks who work on kernel namespaces, aufs, overlayfs and other container used technologies receive no rewards or even recognition while companies like Docker completely reliant on their work suck up the attention and hype. How is this sustainable?
How many know the author of Aufs or Overlayfs? How much support have these projects received and how do these projects sustain themselves.
How many know that cgroups were not namespaced, the issues this created for containers and that they recently got namespaced? How many know about the fantastic advantages but also pitfalls of user namespaces? Isn't this the kind of 'container discussion' we should we having on HN instead of Docker or vendor focussed discussions?
And thanks to the lack of scrutiny, Dockers complex use case of containers has passed unexamined and worse been conflated to containers. This has only increased technical debt for users just stepping in and runs the risk of putting users off containers due to the inherent complexity.
If you want to be popular among developers, the "hello world" developer experience should be extremely simple. See: Stripe, Twilio, Docker, GitHub, etc.
That said, I do think it's too bad that the Docker client/daemon are the lowest-level building block most people are interacting with. It doesn't seem to follow the Unix philosophy of doing one thing well.
> I remember briefly looking into LXC ~6 years ago and being totally lost. Then Docker came along with a simple command line interface and growing repository of images.
I'm not trying to discount your experience in any way, but, remarkably, my impression of Docker was exactly the opposite.
I personally find LXC to be very plain, simple and easy to understand, whereas docker seems intimidatingly opaque and confusing.
But then again, I am a Linux geek :-)
For being just glue, Docker has an unjustifiably large code base, it's quite opaque and a lot of stuff to learn if you want to go beyond "hello world"
If a bash script can do the same (admittedly, using btrfs) using 100 lines of code something is wrong^W hyped up.
Spot on. It's sad how often people behind code that does the real work (LXC, Overlayfs and many more) get overlooked, while marketing-fueled operations get all the glory.
It's also worrying to see that people driving technical decisions are so easily swayed by hype and fail to investigate what's under the hood.
However I do sense (and feel my self at times) a growing upset with the usability/ stability of the docker engine.
The OCI (Open Container Initiative) should allow new solutions to come to market without fragmenting the space. i.e. it should be possible to run any image with any container engine/Linux.
Docker is valued at $1B. It appears that justifying that valuation has taken them into a number of adjacent markets. I wish they were more focused on the Docker Engine itself.
We're also buying into the image-spec for this reason.
Right now the focus is in underpinning CF, so although it is production ready functionally, it lacks the pretty fantastic user experience that Docker brings. Perhaps allowing us to reduce our focus on the containerizer, we'll be able to spend more time improving UX - which would be neat.
I believe the Kubernetes team is working on a simple implementation of the Kube API which is a thin wrapper around runc as well - so you're right, there are some cool things coming around as a result of the OCI.
Hi! I'm one of the maintainers of runC and I wanted to say that the CF stuff around runC is awesome. Keep it up. :D
> I believe the Kubernetes team is working on a simple implementation of the Kube API which is a thin wrapper around runc as well - so you're right, there are some cool things coming around as a result of the OCI.
Actually, ocid is mostly being worked on by people from the OCI community (runcom, mrunalp, myself and others). Though we are getting support from the Kubernetes folks with it, which is pretty cool. :D
I'm going to be giving a talk about rootless containers at Linux.conf.au 2017 in case anyone is going to be attending.
So far we don't see a lot of evidence of our users taking rkt into production over Docker, but I'm very curious if there's a significant number of users like Julia looking to avoid running docker in production.
From a nomad developer standpoint, while building long command strings is a bit awkward compared to calling docker's API, it's quite easy to support new features and manually run generated commands to test our driver's behavior.
So far the command line building is more pro than con. It's so nice to be able to copy & paste commands out of logs to see exactly how rkt is being run.
Though, shoutouts to the rkt guys. They really are awesome and have been helping out a lot in the OCI effort.
Which is an awesome and powerful thing! The ability to define a filesystem without any middle men simply makes some integrations possible.
Thank you and everyone on the runC team so much for your work.
I cannot help myself but that sounds like the worst idea ever and totally backwards. Particularly when they she said that they want to run containers reliably in production, how do they think they are going to do this? The reason why Docker is a "little" bit more than just `Docker run` is so you can run containers reliably and in case of a fail over recover quickly. So how are they planning to run containers reliably in production without adopting an existing, well tested, well working framework?
As I understand it, they keep doing what they have been doing all along, including everything that currently makes their system reliable, and gradually add some of the desirable features of containers in a way they can fully understand.
That doesn't sound like the worst idea ever to me. The danger of ending up with too much of a homegrown snowflake is of course real. You're not wrong about that. But she described the risks of the alternative approaches pretty well I think.
That's not what the blog post says. They don't want to use docker because it does a lot of things. They use rkt because it does fewer things. It's not about learning commands, it's about complexity of software. They selected rkt because it does less stuff, not because it's easier to learn.
I don't buy this, at least as presented. /etc/awesome/blah.xml does not spontaneously come into existence on a host on its own. No matter if you're building a docker image, a VM image, or managing long-lived hosts via a configuration management tool, you have to specify that /etc/awesome/blah.xml is created and has certain contents.
It sounds like the author is working on a poorly structured and documented puppet codebase, where all the dependencies of services aren't clear. Presumably if they can figure them out in order to build working docker images, they could also figure them out and then refactor the Puppet code to make it more maintainable.
At that point they could use their puppet code to generate their docker images, which fits nicely into the "horizontal change" philosophy.
Docker is by far the more mature and adopted development tool. It also runs great on windows, macos and linux.
While k8s can run containers via the rkt runtime, it's still pretty new and will probably introduce unnecessary headache and edge cases.
Docker also has a head start on a vibrant ecosystem for base images.
In my opinion it would be better to focus on docker, and when rkt gets there it shouldn't be much of an issue to switch if desired.
Docker is a developer tool, not a container runtime. The 'containers' it presents require you to commit a daemon on every machine, commit to a weird storage and distribution story when, you know, files of containers served in a flat directory are totally adequate, and so on. Docker made extremely complicated choices for a lot of things and now everybody wishing to advance containers has to deal with presuppositions like these, where Docker exists and the motivation for not using it is unclear to a lot of people, you included.
The fact that you don't like Docker doesn't change any of that. If you don't like it, don't use it. But there is no justification for you to claim it is only a developer tool when it is obviously more than that.
Docker is tool to interact with LXC (and hopefully other container systems).
FreeBSD can run 64 bit and 32 bit Linux binaries via it's Linux emulator[1]. Plus FreeBSD has had containers since FreeBSD 4 which was released in March of 2000, except containers on FreeBSD are called jails.
> If MacOS doesn't have certain linux kernel features, neither does FreeBSD.
Just because one operating system doesn't have a feature that does not mean a different operating system does not have that feature.
Edit: My viewpoint was focused more on the composability aspect of containers. But I stand corrected otherwise.
Thats incorrect both jails and "containers" are examples of OS level virtualization[1]. Instead of virtualizing the hardware you virtualize the operating system, ie system call table.
[1]: https://en.wikipedia.org/wiki/Operating-system-level_virtual...
- rkt can run any docker image in the vibrant ecosystem: https://coreos.com/rkt/docs/latest/running-docker-images.htm...
- Help test Kubernetes and rkt with minikube, it is working nicely today: https://github.com/kubernetes/minikube#using-rkt-container-e...
I believe that Docker bet the company on the wrong business model. Therefore knowing that you can run containers without Docker can be desirable, as it gives you options in case Docker turns out to be not viable.
Docker made it easy to build and run containers from the command line. This technology was released at the right time and in the right place. And it got huge traction. Docker used this traction to get a very large amount of VC capital. Therefore they now need to show a very large return.
How do you go about achieving that large return? A common way to do this is to use one piece of technology that is popular as an anchor to lock customers into an entire stack. And then you monetize the stack, which is easier as it has higher value. The technology in this case is the container runtime, and the stack would be an full enterprise grade container orchestration system.
This strategy worked for others. Microsoft was a master in this. VMware as well, and so is Oracle. The reason I believe it will not work for Docker is:
1. The anchor technology is open source. Microsoft managed their lock-in via Windows, VMware via ESX, Oracle via their DB. It worked because those are proprietary. In the open source world you can just take Docker.
2. Even though the technology is open source, you can still try the ecosystem approach where you exploit certain network effects attached to your distribution of the code that protect you (e.g. vendor certifications). This is what worked for Red Hat. In my view it is unlikely to work for Docker. I don't think certifications are required as most container apps today are not vendor apps. And I would not know what else they could use.
3. For orchestration solution they are engineering against Google's Kubernetes. With all respect for Docker, this is going to be very hard as Google has a huge amount of experience in this area.
4. Due to Docker's need to protect their business model, there is no true community in the sense that there is an open exchange of ideas. The docker agenda prevails over technical decision making (like: should you have a docker daemon? Should Swarm be part of the Docker Engine?). Contrast that with Kubernetes which run in a very open manner by Google. This is why OpenStack got the traction, and why Eucalyptus and CloudStack died off. OpenStack had the most open community.
All of this can lead, IMHO, to a situation where Docker will not succeed in delivering Swarm as they see it. Kubernetes will become the default container orchestration system, and Docker is relegated to a small place in the stack. This situation would make it hard for Docker to justify their valuation.
Docker has realised the money is in the platform. So they're competing against Red Hat, who have OpenShift.
They're also my employers, Pivotal, and IBM, and HP, and SAP and every other company with a Cloud Foundry distribution.
The problem for Docker is that Red Hat, Pivotal et al started with selling an entire platform to F1000s. Docker and Kubernetes have great groundswell in the community, but the difference is that Docker need to make money from a sold platform and Google don't -- their play is GCP.
> For orchestration solution they are engineering against Google's Kubernetes.
Let's not forget Mesos, Nomad, Diego (the Cloud Foundry bits) and so on. And yes, it turns out this problem is actually very hard. Much harder than it looks.
> Due to Docker's need to protect their business model, there is no true community in the sense that there is an open exchange of ideas.
This is one of the reasons Pivotal donated Cloud Foundry IP to the Cloud Foundry Foundation, and it's why the Foundation rules are written to grant voting rights to members which contribute engineering effort. Red Hat are relevant as a large and critical part of an ecosystem. So it is for other successful OSS companies. But being the "owner" tends not to go as well. People who buy from us want to know that they can switch to IBM without much disruption and vice versa.
> This situation would make it hard for Docker to justify their valuation.
I can't blame them for trying. And I suppose they didn't see Kubernetes coming to suck the oxygen out of a big part of their platform.
The weird story of Docker will be of a company that created a PaaS, abandoned it for a component, then tried to build a PaaS.
Disclosure: as I noted above, I work for Pivotal. As Docker moves into platforms, we are competitors. Though we cooperate on runC.
> What is the motivation and benefit for running containers without docker?
I regularly hear from people who want to run containers without Docker. There are several motivations, all of which are perfectly valid:
1. Learning. It's fun to build things from scratch to understand how they work under the hood.
2. Bad experience. Early versions of Docker were quite buggy, and we initially struggled to keep up with the colossal growth in usage and feature requests. As a result, many of the people who tried Docker in production too early were badly disappointed. Some of them decided Docker wasn't for them, and started looking for alternatives.
3. Extremely custom use case. If your deployment is larger, or more complex, or more specialized than 99.99% of deployments out there, then "mainstream" platforms like Docker might not be the right fit for you. Of course we try to make Docker as customizable as possible, to support more a wider spectrum of use cases with plugins. But realistically, no single platform can cover all use cases, and I don't think any platform ever will. Docker is no exception.
4. Philosophy disagreement. Different people have different opinions on how applications should be developed and deployed. Docker tries very hard to be agnostic - to accommodate as many opinions as possible. But we can't please everybody. If Docker does not fit your philosophy of development and deployment, then the natural response is to look for an alternative.
5. Competition. Many Docker competitors started out as extensions or modifications of Docker, and over time are looking to reduce their dependency on Docker.
I'm probably missing other reasons, but these are the ones I've been most exposed to. These are all reasonable reasons to not want to use Docker.
Our approach is that, if you want to run containers without Docker, we should make it as easy as possible. In practice that means spinning out as many of the underlying components as possible (what we call the "plumbing") so that you can assemble it yourself without being stuck using the entire Docker platform. For example:
- containerd [https://github.com/docker/containerd] is our low-level container runtime.
- runc [https://runc.io] is a standardized "container executor", which we donated to the Linux Foundation as the reference foundation for the OCI spec.
- libnetwork [https://github.com/docker/libnetwork] is the low-level networking implementation (including overlay networking which is a very useful primitive for container clustering)
- swarmkit [https://github.com/docker/swarmkit] is a clustering/orchestration implementation
- notary [https://github.com/docker/notary] is a cryptographic content verification tool, which you can use to sign and verify container images.
- infrakit [https://github.com/docker/infrakit] automates the provisioning of infrastructure capable of running containers.
Our opinion is that, even if you don't use Docker, by using these components for your own purposes, you are indirectly contributing to making Docker better. Splitting out these components has also forced us to refactor Docker into a more modular, more robust design.
We even send Docker employees to explain how to run containers without Docker :) For example here's a talk we gave at Linuxcon: https://linuxconcontainerconeurope2016.sched.org/event/7oHM/...
Great concept, poor execution.
At the moment, Kubelet needs to control the Docker daemon (a somewhat brittle relationship), and a bunch of concepts — from networking to volume management — could be simplified by making the container runtime more K8s-specific and less general-purpose. A tighter coupling would also mean you'd always be running K8s against the exact container logic it was made for, as opposed to now, where the loose coupling means you'll be subject to bugs (and there are so many) in new Docker versions.
That and the fact that Docker is increasingly building in competing concepts (services, etc.) via swarm mode, which I don't care for.
Docker still has a place for things like image building, of course.
Personally I think the long-term solution is for kubernetes to adopt containerd - it's the small, stable, rock-solid container runtime that it needs.
Whether the Kubernetes community continues to build primarily on Docker, or switches to containerd - either way we will support them and help in every way we can. Behind the superficial internet drama, there is a solid engineering collaboration between both communities, particularly between Docker and Google. We may disagree on a lot, but at the end of the day we have a lot of users who rely on us, and we have a responsibility to work together to solve their problems.
That's quite inaccurate. It's is like saying that OpenStack started out as an extension to KVM. Both had very different (original) scopes.
> Whether the Kubernetes community continues to build primarily on Docker, or switches to containerd
Kubernetes has 3 container runtimes at the moment: Docker, Rocket, and the new remote runtime which allows you to plug in a separate server using a protobuf interface. There is a lot of focus on the remote runtime now. The CRI-O project for example is implementing a remote runtime that allows any OCI conformant runtime to be plugged into Kubernetes.
The focus on diversifying away from Docker is very recent, and is the direct result of the competitive tension I talk about earlier in the thread. It's completely understandable.
EDIT: A perfect example of that competitive dynamic is actually this very thread! the GP (`sysexit`) works at a competing vendor. You can tell in his post that de-emphasizing the dependency on Docker is important to him. In fact that talking point is so recognizable that I knew where he worked before even looking him up. There is nothing wrong with that, every vendor has an "official party line", and Docker is no exception. The point is, this is a delicate issue which has more to do with inter-personal and business dynamics than with code. And you should definitely verify the credentials and agenda of everyone speaking of authority on the topic of containers platforms. We vendors are everywhere, and we are biased as hell.
As a Pivot, I can confirm: I am biased as hell.
I think you miss a reason at least equally as important, if not more: There are a lot of people who want to do non-container runtimes but still leverage the rest of Kubernetes. I don't want to say no to them (who am I to decide what tech is "good"?) but I don't want their code in my tree (too big already, too hard to maintain, too much email, etc).
Ergo: plugins.
But you're right, kub itself is a neatly generic orchestration component which can be used with non-container things. That is an excellent reason for decoupling the underlying container runtime.
The experience of the Garden API (a containerising system that ultimately predates Docker) is that tighter coupling is actually a mistake.
If Garden had been tightly bound to its implementation or orchestrator, it'd still be using raw Linux system calls and running under a 2nd-generation orchestrator.
Instead it's now using runC, OS X or Windows APIs to create containers and runs under multiple consumers -- most heavily under Diego, a 3rd-generation orchestrator and Concourse, a project automation tool.
Disclosure: I work for Pivotal. Many, but not all, of the engineers who've worked on Garden work for Pivotal also.
Mad props.
Great! There is one question I would like to have answered, by you and you alone:
if I'm using SmartOS, and therefore zones, and all my applications and configurations are packaged into OS packages, and I use imgadm(1M), vmadm(1M), and Smart DataCenter (SDC), what do I need Docker for?
Fully isolated, virtualized UNIX servers. Servers are cattle, not pets. Deployments in up to 25 seconds.
That was the challenge. And it was fun to solve.
Have fun maintaining OS packages.
Oh I am! It's incredibly satisfying to be able to run systems where no human has ever logged into, and no file has ever been manually modified, and all the files and all the changes are accounted for by the operating system.
See also, not quite the same, but very much related and relevant to your comment:
http://www.perkin.org.uk/posts/building-packages-at-scale.ht...
https://blog.nimbleci.com/2016/10/11/how-to-deploy-jenkins-c...
https://news.ycombinator.com/item?id=12750685
Configuration management at scale, using an operating system's software management subsystem is the old, new-new thing. It is the foundation and the future of even more massive cloud computing. Now you'll be able to say someday in the future: "ha! I read that on Hacker News years ago!", and point to this post.
Anyway, someday in the future I might look back on this and think "ha! someone put all that effort in even though NixOS already existed".
All I did was ask a question.
Is critical thinking, even just asking questions, undesirable now, unless someone is being praised to high heavens? Is that what we're down to now?
"ha! someone put all that effort in even though NixOS already existed"
SmartOS virtualization is based on Solaris zones, which existed long before people on GNU/Linux even realized that containers were the way to go. The groundwork, and my own work was laid down long before any of this became hype. Project Kevlar, nee zones was started around the same time as NixOS, and another participant already mentioned that large companies did large scale deployments and configuration management via OS packages, which is both true and correct.
Docker is just a simple wrapper that reduces the amount of knowledge needed to run a containerized process. It did not become hype because of any technological advantage, it became hype because this old technology became accessible not just for the kind of people/companies that don't (can't) run enterprise class Solaris deployments, but also the kind that don't have 20 years of experience in making robust well managed deployments.
Your process might seem easy and straightforward to you. But for the average developer and Linux enthousiast, there's not much that can the simplicity of the Dockerfile. Systemd or a more specialized system like Kubernetes to run the containers, and a simple script to provision your machine is all you need. You can pick all of this tech up in a few days and steadily grow into more advanced subjects like monitoring, logging, security.
Anyway, you're not helping anyone by just saying "SmartOS is better than Docker+(list of redundant and unrelated other tech)", maybe instead show us a one-page tutorial on how to set up a SmartOS system with a webapp stack (i.e. LAMP, MEAN or RoR) that can be executed by a newbie without a creditcard, registration, trial or phone call.
Yes I did.
You asked a question leading into a lecture on the technology you prefer.
Only after you left your Good luck with packaging everything, which I have good reason to believe was cynical sarcasm.
It did not become hype because of any technological advantage, it became hype because this old technology became accessible not just for the kind of people/companies that don't (can't) run enterprise class Solaris deployments,
Performing configuration management at scale with OS packaging has nothing to do with Solaris - it works on any operating system which has a Bourne-family shell, AWK, and sed.
but also the kind that don't have 20 years of experience in making robust well managed deployments.
Then it would be a good idea for such people to actively seek mentors.
But for the average developer and Linux enthousiast, there's not much that can the simplicity of the Dockerfile.
That's what distinguishes the average developer. A true enthusiast will go out of their way to become a master of the art, so the assertion doesn't apply to them.
It's easy to hack several hundred thousand files together and deliver them via Docker. But, what happens when it's time for upgrades? Go through the entire hackfest again? Every time?
Anyway, you're not helping anyone
that depends on what your mentality is: if you're a true enthusiast, you're probably reading a lot, and when you stumble upon crumb trails like mine, the light bulb might go off reading what I wrote, and it might just be the clue you need (that's the idea). And you wouldn't be the first or the last person that happened to.
As for me, I'd gladly show how to do what interests you, only I have no blog, and no time to set one up, fight spam, hack attempts, and so on. I can afford one short answer like this, but I cannot afford months of engineering implementing the infrastructure for a blog and hardening it.
And by the way, nice stab at Joyent on your part, but SmartOS is free, open source software, and the entire ecosystem + the 14,000 packages never cost a penny. Even their cloud on a laptop ("COAL") preconfigured image is gratis.
As for packaging being hard... there is such an abundance of packaging documentation for any popular operating system... all you have to do is sit down and "warm up the chair" by reading it. Do you think I learned it some other way?
which I have good reason to believe was cynical sarcasm.
Then it would be a good idea for such people to actively seek mentors.
Do you think I learned it some other way?
Gladly accepted.
I have done some packaging for Debian, CentOS and recently Alpine and whilst the systems are smartly designed and effective, using them is neither easy nor enjoyable.
I cannot write anything of Alpine Linux, as I have never used it, but I have done Debian / Ubuntu packaging and configuration management with packages, and that, compared to RHEL / SLES / SmartOS / Solaris / HP-UX / IRIX, was the hardest to get right.
One of the reasons why Debian and Ubuntu are so hard to get right is that the Debian packaging guidelines are insane: for example, it is not allowed to have packages named in any way other than lowercase; there is no /opt, as the Debian people do not understand the filesystem hierarchy standard and consider it a hindrance, for reasons completely unknown to me.
So now imagine you had to build, say, your own PHP with OCI8 support: you cannot name the package php, nor can you deliver it into /usr, because the next upgrade from Debian could break your production. And there is no /opt, so neither vendors nor other 3rd parties have a place to safely deliver their software (/usr/local is not it, and is against standards).
What I'm trying to say is, you picked one of the worst operating systems, ever. Even RHEL, as bad as it is, is nowhere nearly as poorly designed as Debian GNU/Linux.
Pick a better OS. pkgsrc format is pretty straightforward, and AT&T SVR4 packaging[1] is the absolute best when it comes to doing configuration management. Unbeatable.
This might not be your cup of tea, obviously it wastes system resources, but it does simplify the ordeal.
If delivering software which JustWorks(SM) is an ordeal, then you need to step back and seriously re-think what needs to be corrected. Delivering software should be a breeze, and it should happen automatically, meaning that the software management subsystem of the OS should worry about it for you. But that also requires you to seriously revise how you put software together, because if it's an ordeal for you, something is majorly wrong. (Since I don't know your situation, I cannot comment what.)
Or they could use Docker and focus their energy on building value for their company so at some point it can afford to employ proper system administrators. Who then may or may not decide to migrate the systems to SmartOS.
This line of thinking, is, in my experience, completely, utterly incorrect on several levels:
developers should not act as if system administration is beneath them. One can never, ever become a master developer without becoming a top-notch system administrator: how can anyone develop software which is correctly integrated and runs on an operating system, if they do not understand how to administer that system? I have seen that over and over and over again in my decades of experience, and I have seen no developer who was able to code quality software which the system administrators didn't hate, because it was all hacked up and required lots of hacking to get working.
I just think that if knowledge-intensive engineering can be avoided it is worthwhile. Especially when an operation is young.
If you do that, it will come back to haunt you. With a vengeance. And the reason it will come back to haunt the company is because it's taking on a huge technical debt.
Are you familiar with the concept of taking on technical debt?
Thank you for saying this! Building OS packages and OS images and provisioning servers was done automatically 10 to 12 years ago with much less work.
It's still the preferred choice for many large tech companies and nobody talks about it because it just works and it's nothing new. Fast deployments, immutable infrastructure, modularity.
So, my motivation of running without docker would be to understand every little thing they do to actually be able to use them and rely on them.
sudo usermod -aG docker $your_user
There ya go.
In newer distros its as easy as apt-get install docker.io
However. If what you want is to simply understand what's going on under the hood, this is a great way to do it. The only thing better that I would recommend is to actually make your own Docker. It's not that difficult, you're basically just slapping together some system calls and execvp()ing some other standard tools, which is what Docker does, and you don't need to support any of the extra fancy features. But you gain an intimate understanding of what could be going on when stuff in production is breaking.
Not sure if you've seen this, but the author wrote a great post about the Linux primitives that Docker is built on: http://jvns.ca/blog/2016/10/10/what-even-is-a-container/
Just trying to understand why rkt is less overhead...
The docker daemon has historically had some stability issues as well as some security implications. Running a command line tool like rkt is a vastly smaller attack surface and less complex stack overall.
You are namespaced, so the linux kernel promises that even though you're root, you're not dangerous, and there is syscall filtering and shit going on.... but that historically has not really fared that well!
But your statement is false. You're root with and without privileged. Privileged gives you back CAPABILITIES which are different than USER, so your claim is bullshit.
That is objectively false.
uid = 0 is "privileged" basically everywhere in the kernel, from filesystem management (reading a file bindmounted in that's owned by root e.g.) to binding to low ports (like 80).
As you can see from the docs, it says "the most important security improvement is that, by default, container processes running as the root user will have expected administrative privilege (with some restrictions) inside the container but will effectively be mapped to an unprivileged uid on the host."
This implies the reverse, that if you don't use userns then your process as root in the container will be mapped to a privilege uid on the host.
This is all I'm saying is true. You clearly don't understand what I'm saying.
This API has to enabled explicitly. Docker daemon works by using a unix socket instead ( "/var/run/docker.sock" ).
> acts more or less like an init for containers
There is Docker daemon and there is Docker CLI. Both have separate scopes.
Docker CLI is glorified curl, everything happens in the daemon (containerd being logically -- but finally not physically -- part of the daemon).
If you'll indulge me putting on my corporate hat for a minute: we've had some customers very happy scheduling a high volume of containers at a high rate on nomad:
- https://www.hashicorp.com/c1m.html (mentions a docker bug even)
- https://www.youtube.com/watch?v=MRtRwhL5lwM
If at all possible I'd recommend using our Java or exec drivers as they use builtin containerization and avoid the overhead of docker or rkt.
Since then, Docker has improved quite a bit on these so the most in your face practical differences are smaller, but there are still philosophical differences that affects it.
E.g. rkt comes out of CoreOS. CoreOS does a lot around embracing systemd to its full extent. Systemd can provide a lot of the capabilities that Docker did itself, and parts of rkt's design flows from that. E.g. restarting, querying status, capturing the logs, so in a systemd based system, Docker integrated fairly poorly in that systemd would be starting and keeping track of a Docker client rather than the process actually controlling the container, while rkt fits right in.
Again, the difference is getting smaller, and various tools like runc etc. from Docker now allows you narrow the gap even more (if you put in extra effort).
Try both, basically - they're similar enough that it's worth figuring out which "flavor" you like best.
It's also how Hadoop and other m/r work.
:s/build a comtainer image/build a container image/g
to fix a spelling error
https://github.com/coreos/rkt/blob/master/Documentation/deve...
Clever commentary trap: why not? Here's why. Oh, well, nobody's forcing you to use it.
> Docker is a developer tool, not a container runtime.
It's just not true, and that's why I responded.
It was not intended for production usage from the beginning, as you claimed, but I'm already tired of responding to this thread because I'm turning instantly gray for not buying into Docker so why would I bother proving that you're not correct?
> Containers existed before Docker and will exist after Docker, and articles like these are desperate attempts to remind everyone of that in the face of Docker's unilateral destruction of the concept. I know I'm right because of comments like these that presuppose "well, if you're doing containers, use Docker, the alternatives just aren't there." Except they were, before Docker, and after Docker.
> Docker is a developer tool, not a container runtime. The 'containers' it presents require you to commit a daemon on every machine, commit to a weird storage and distribution story when, you know, files of containers served in a flat directory are totally adequate, and so on. Docker made extremely complicated choices for a lot of things and now everybody wishing to advance containers has to deal with presuppositions like these, where Docker exists and the motivation for not using it is unclear to a lot of people, you included.
See the first sentence in the second paragraph? That's you. You wrote that. You did not present it as an opinion, but as a fact.
> Docker is by far the more mature and adopted development tool.
That's him. He wrote that. I added to it with something that you disagree with, and I'm rapidly tiring of interacting with you because you're making it extremely hard to remain civil.
But since you're the expert on Docker's production intentions, could you perhaps discuss Swarm and how it compares against competitive technology in the field? We can start small: what kind of scheduler does Swarm employ? Two-level, optimistic? What is your understanding of the runtime and performance bounds of the selected scheduling strategy? What is the expected latency for scheduling decisions as the number of executing containers grows? How does Swarm handle failure to schedule?
Could we then compare that scheduler against Aurora, Marathon, Kubernetes, Omega, and Borg? Why do you feel that Docker is production ready in light of the competitive work being done in this space? What do you feel is the difference between Mesos and Aurora? Between Docker and Kubernetes? Since Docker is intended for production usage, can you elaborate on some of the challenges you've experienced running it in production?
The title is "Docker: the Linux container runtime." So, your assertion that "Docker is a developer tool, not a container runtime" is simply false.
The text clearly describes deployment of docker containers as much more than simple development tools: "docker can run on any x64 machine with a modern linux kernel - whether it's a laptop, a bare metal server or a VM. This makes it perfect for multi-cloud deployments." That sounds like production deployment to me.
Whether or not you like Docker, or think it works well in production, is immaterial to this. Docker is a container runtime, it is intended for production deployment, and that means it is not merely a development tool as you claimed. The first website Docker ever published is proof of Docker's intentions. It was intended as a production tool from the very beginning.
Docker wasn't 'recommended' for production until 1.0: https://blog.docker.com/2014/06/its-here-docker-1-0/
That is not the same thing as saying that Docker was never intended to be used in production, and was only ever intended to be a development tool. That's the view I'm arguing against, and it's not trivial -- it denies the fundamental purpose of Docker.
Are you sure you replied to the correct post?
twblalock seems to me to be the calm but persistent voice in this detached subtread.