Are Containers the Right Answer to the Wrong Question?
open.citrix.com
open.citrix.com
And SVR4 and SCO binaries too for that matter.
The idea that your kernel can only give you one syscall-API is so 1990-ish.
perhaps jails need's a little boyish mustache?
also; hurrah for jails.
The only my wish for FreeBSD is some easy way to do pkg install $PKG --and-run-it-in-jail=jail22 --thxbye
If you're talking enterprise use, sure, some app servers might need more than a few GB, but that's a memory problem, not strictly a bitness problem.
One program would spin up lots of threads, each with a 1MB stack, which was hardly used. It'd die on 32bit even with no real memory pressure.
Also, 32bit is less likely to get as much testing for server apps.
And, of course, you know much better what would happen to CPU caches and I/O context switching in a host OS under even a moderate load.)
Update:
Dear anonymous down-voters, I ran a FreeBSD Jail-based hosting in production for years in 2000s, while you, probably, didn't even finished a secondary school.))
Jails are not comparable to Docker. Jails are the container technology, like LXC and OpenVZ, which Linux has had for almost a decade.
Now, could you elaborate how Docker is not a "container technology".
What Docker provides is the level above - the layered images, repositories, automated services discovery, etc - which LXC/Jails/etc don't provide.
A closer equivalent in the *BSD world would probably be projects like CBSD, ezjail and such (in terms of the stack layer, not in features).
What I am trying to say is that chrooted envs for process segregation (like running a separate mysql instance) is not such new and exciting idea, and it has more disadvantages than benefits, at least for production use. (It is substantially slower and unpredictable due to memory access and I/O bottlenecks).
On the other hand, for 98% of Docker = "containerization" (a new shiny meme) = cool, without even a bit of understanding what LXC is.
Is now a good time to explain why SysV shared memory syscalls were never wrapped and are thus shared across all jails in FreeBSD? :-)
That said, there are much deeper questions here than 'container good or bad?' Not surprisingly, they are nearly the same as 'compilers good or bad?' that were all the rage before.
So hypervisors and containers are the same two systems, hypervisors abstract to the hardware level and containers abstract to the OS level. By setting the level of abstraction they control what can be shared and what can't, and what differences can be accommodated in the same system.
At scale, efficiency is associative. Which is to say that you can be more efficient in your use of resources (higher utilization) when you associate similar abstractions together. You do that by maximizing the commonality. So if the only common thing is 'runs on x86 hardware' that is your common point. If its 'runs on Linux' (or Windows, or Solaris) as a common point then that you get some efficiency by running one copy of the OS and multiple processes in that OS. If the common point is 'Rails' or 'Go' then you can gain efficency by keeping hardware, OS, and software stack in common.
The further up the abstraction level you go, it loosens constraints on the bottom layers while increases them on the top layers (you have to write all your code in Rails, but you can run on x86-64 or ARM-64 architecture machines, versus you can write you code in any language/system that targets x86-64 hardware) Different organizations can dial that knob based on their ability to be flexible. So organizations with a lot of legacy binaries targeting x86-64 they really need hypervisors, but organizations where all of their code base targets Linux, they are much better off with containers. At a small scale the numbers are a toss up, at Google/Walmart scale the numbers become company controlling.
TL;DR: this is timeshared mainframes all over again. I'll dust off my pipe and white technician's coat.
In the beginning there were batch run mainframes, and it was good.
And Lo!, did the bean counters notice thy ever under utilized CPU cycles. So became time sharing mainframes.
The masses rejected the cold noisy cave of the server room and so it was decreed that the terminal shall forever more be remote.
Thy ever indiscriminate shrink ray doth break the bonds of the mainframe, and set forth the world into mini computers. "Hark!" said the masses; we demand the Personal computer.
The masses were mollified, until the becoming of cheap internets. "But wait!?" they cried we require some sort of network computer.
Time passed, and a tower of babel was conquered by XML, JSON and eventually the cloud.
http://english.stackexchange.com/questions/38741/use-of-and-...
The article was reiterating the advantages of virtual machines -- which handles different operating systems.
I thought the timeshare concept on mainframes was user-&-application-level time slicing and not os-level. (E.g. an IBM 370 does not simultaneously run OS/360 and MVS operating systems in isolation.)
Therefore, the closer analogy to mainframe timesharing would be Citrix Xen desktop or Microsoft Remote Desktop Session Host.
You can run lots of code from the 70's with little to no modification on a z series mainframe, because its visualized.
However My analogy is based on the idea the cloud is effectively a mainframe. They are espousing the joys of single application VMs that effectively rely on the host to do the time-sharing. Something very reminiscent of early timesharing OS
Ok, I thought the main mechanism for running old code was instruction-set-compatiblity-mode emulation and not full isolated virtualization. Could you copy an entire OS+app stack as a self-contained disk file from one mainframe to another?
Regardless of the z series vm capabilities, I don't think people think of that as "mainframe time sharing" and that's what tripped me up.
The best part about IBM mainframes was/is that you shouldn't need to worry about copying the OS to keep the App happy.
The mainframe OS effectively provides a VM in a similar style to python or java.
You tell it what version of machine its supposed to run on and it'll provide an environment with all the quirks.
The article goes on to talk about uOS type systems, which are there to provide a Linuxy shim (with all the quirks) between the hypervisor and the App. (do you start to see where I'm going? :) )
It solves a lot of problems in small to medium companies; it's an answer to a desperate call for help of sysadmins too tired to hunt 15 years old cpu fans on ebay, and managers hearing always the same infernal stories about what it's like to do anything down in the basement.
It solves the problem of the same small to medium companies having to buy new hardware for each application they buy, usually because the vendor tells them to do so. Many software started to get shipped as a whole OS stack, because that was easier for the vendor to support.
It solves many other problems, but let's stop here. These problems are going to stay, and so will virtualization.
However containers offer a way to isolate your environment so that they can survive longer and coexist better with other apps. If you build your environments using those building blocks, then after 10 years, you'll have a slightly smaller mess (which you can still handle with virtualization).
If you really have a lot of machines then, you need to have a good system for managing your hardware, for bulk installation etc. You have to manage new kernel and container management software rollouts in more or less the same way you'd need to manage hypervisor rollouts.
But since you have a lot machines, which means that you encounter hardware failure more often, the relatively higher frequency of kernel and container management software updates has still a smaller cost than virtualization overhead (provided you have enough machines).
Or put in another way, virtualization overhead is negligible if you're a small fish. It's cheaper for you to buy more machines or a stronger machines if all you need is to cover for that overhead.
For example, I have a nasty little calendar script that I have to move. The rest of the app is long dead, but this one little worm keeps on biting.
It uses regex to parse XML. (insert cuthulu joke here) its based on php5.old I need to move it to 5.new+secure. The OS has an update function that is know, tested, documented and works. I can install a new RPM/deb and be fairly sure it'll work if it installs
Standalone apps, in the way that people are building them wont. Conatainers rely on the underlying operating system for various things. Any exceptions are bundled with the app.
a true VM is free to be moved to a new host, a container is not. You are hamstrung to that machine and possibly that config.
So If you built the farm on docker, you're stuck with it, on that hardware. unless you want down time, or was to re-engineer things
I don't understand. I assume that for "underlying operating system" you mean the OS that hosts containers. Unless you depend on some OS specific things like iptables or stuff like that, it should be trivial to rebuild a container (e.g. make a new docker image with a new php) and test it.
Containers are not so great if you migrate into container software that has not been packaged for containers in the first place. It's not that it won't work, but it's easy to find example where you could run into troubles.
The whole point of solutions like docker is to give you the consistent environment that will help you to test your changes in isolation.
> [...] The OS has an update function that is know, tested, documented and works. I can install a new RPM/deb and be fairly sure it'll work if it installs
Until you have a distro that's too out of date for that to work properly. The same thing happens if you use containers with too much of an "OS" in it. There is currently a lot of debate about what's the right balance between microcontainers or full fledged mini-OS inside a container. On one end of the spectrum you move your problem to container orchestration, while on the other you risk breaking the isolation abstraction provided by the container by running an OS that doesn't have full control on it's environment (e.g. kernel, disks etc) and thus breaking the guarantee that existing upgrade tooling will work flawlessly.
> a true VM is free to be moved to a new host, a container is not. You are hamstrung to that machine and possibly that config. > So If you built the farm on docker, you're stuck with it, on that hardware. unless you want down time, or was to re-engineer things
That's true, only a VM will guarantee that an arbitrary legacy application can be moved to a different physical machine in face of arbitrary incompatible developments in the underlying hardware. Well, if you want to stretch that to the extreme, only a full emulator can guarantee that you will be able to run your legacy things on a completely different CPU architecture.
However, you must be aware that VM offer you a way to acquire a technical debt, which is great in that it allows you to decouple your immediate hardware upgrade needs from your software reengineering, but you must be aware that you'd have to pay off this debt.
I have a 15 years old mandrake linux VM with some legacy software. It's well known, tested and documented update system won't help me maintain the mess that's inside that VM.
http://www.rightscale.com/blog/cloud-industry-insights/googl...
The overall concept is not without merit, the OSv site (http://osv.io/resources/) has some more interesting material. I'll still be sticking with Docker though.
I found it to be a very nice, powerful, and enjoyable language.
Projects like MirageOS and ErlangOnXen are fantastic and I hope we see more of them, but getting to the point where these kinds of runtimes are viable for most people is a long road.
In general, I see things like Docker/Mirage/Xen to be complementary, rather than antagonistic. A good example of this is Google's Docker support, which is via a custom VM.
Side note: Why use a Twitter short URL when the original [2] is pretty succinct anyway?
[1] https://twitter.com/ID_AA_Carmack/status/476046680698003458
Thanks for your reply, don't have time to invest that much in unikernel, I just keep an eye on it. But the use case is very appealing to me and to you according to your writings on your blog :) I'll keep in touch if I investigate further the Unikernel way. Thanks for the John Carmack's tweet, very interesting, and a validation of our intuition by one of the greatest programmers :)
Side note : I sent the comment from my smartphone and the quickest way to retrieve the link to the ACM article on Unikernel is by browsing my twitter feed :) hence the twitter short URL
Finally, and most importantly, your operating system does an awful lot of stuff that is not trivial, not in the slightest. The Linux kernel has better page cache algorithms than your unikernel, a better networking stack, vastly better hardware support, etc. Plus tons of people thinking about how to make it better every day. No one is going to port that to OCaml.
In this case it is not a application though. It is a runtime environment... something in between a whole VM and a container?
Who knows, maybe Citrix has something to sell that makes the author's opinion very "important"?
Just apache's virtual hosts isn't good enough anymore, customers want Rails, Nodejs, Java for their $20/mo.
Virtualization for real-world production is nonsense, especially those idiotic setups like Java under Linux under VMWare under Windows (you wouldn't believe how many "professionals" are cock-sure that this kind of setup is "robust" and most stable.)
Jokes aside, Docker is eating VMWare's lunch being a fresh new meme promising miracles (yeah, problem of cache trashing and context switching for I/O in a host OS will surely be solved by someone else.
[Cloud Operating Systems] enable the vision of light and fast application servers, while preserving the superior security and flexibility of true hypervisors.
As an extreme case, consider running all of your applications on your Desktop in their own VM. Now, you are in your web browser and you download a Word document that you want to edit. Somehow, you have to transfer this document to the Word Processor VM to edit it (they have different file systems so you need to use something like scp). If, after you're done editing it, you need to send it to someone via email, then you'll have to copy it to the Email VM. It's not how to see how this seriously impacts usability!
When sharing is required, then a Hypervisor is not helpful. That's not to say that Hypervisors are useless, as the author claims his opponents say. Hypervisors are good for running different operating systems on the same hardware and they are good at isolating the different OSes from each other. However, once any sharing is required, hypervisors are not the right tools for the job.
Edit: As for your extreme example, I believe Bromium (ex-XenSource founders) is doing what you describe.