Docker for Mac and Windows Beta
blog.docker.com
blog.docker.com
So, buyer beware, especially if said buyer also uses tools like Vagrant.
[1] https://github.com/mist64/xhyve/issues/5
I've said before that I think the Docker devs have been iterating too fast, favoring features over stability. This development doesn't ease my mind on that point.
EDIT: I'd appreciate feedback on downvotes. Has the issue been addressed, but not reflected in the tickets? Has Docker made changes to xhyve to address the kernel panics?
Is this type of comment discouraged on HN? If so, why?
Yes, any new virtualization projects should strongly consider using it, but that doesn't mean any issues with projects with have a massive investment in tooling and ecosystem should be considered deprecated and dead just because of this.
Folks are welcome to disagree, but Docker has a history of shipping software which uses a 3rd party feature which breaks, to which they frequently responded "not our code, talk to someone else": btrfs instability, corrupted volumes due to conflicting devmapper libraries, iptables dropping routes, upgrades orphaning containers, etc.
I realize they don't have control over all of the variables, but constantly releasing unstable 3rd party features was not the greatest behavior, and the "Not My Problem" response to issues is aggravating.
All that said, since they're working against their own fork of xhyve, it is a sign that these kinds of issues will be addressed by the Docker team this time, which is a good thing.
I doubt it would be getting downvotes if the comment was just a statement of fact without the somewhat random slam against the Docker developers.
If anyone sees any host panics ever, we'd like to know about it (beta-feedback@docker.com) and fix it in Docker for Mac and Windows. Fixes range from hypervisor patches to simply doing launch-time detection of CPU state and refusing to run if a dangerous system condition exists.
I personally discovered this bug while using dlite, which uses xhyve behind the scenes.
xhyve is wonderful but still needs some work and it seems like the main dev isn't interested in continuing work on it at this point[1]. Hopefully Docker's usage will spur more work on xhyve.
[1] Last commit on xhyve is December 28th, 2015 https://github.com/mist64/xhyve/commits/master
Makes sense, that's fine. But, it seems they've gone (and continue) to do that very thing!
> Volume mounting for your code and data: volume data access works correctly, including file change notifications (on Mac inotify now works seamlessly inside containers for volume mounted directories). This enables edit/test cycles for “in container” development.
This (filesystem notifications) was one of the major drawbacks for using Docker on Mac for development and a long time prayer to development god before sleep. I managed to get it working with Dinghy (https://github.com/codekitchen/dinghy) but it still felt like a hack.
At this stage, pointing it to the weirdest and most wonderful filesystem stressers you can find is welcome. We'll leap on any issues you find...
Intel's HAXM doesn't (seem to?) play nice, and asks for an exclusive lock. See https://github.com/mist64/xhyve/issues/88 and https://code.google.com/p/android/issues/detail?id=197915
Is there some kind of caching? If yes, what is the impact on tools like make when checking atime or mtime? If no, is there a perceptible impact on latency, for example when compiling a large project in the container?
I have an open bug against docker-compose (docker doesn't do the same thing by itself) where the wrong layers are being used, but only on virtualbox.
Hopefully this will solve that problem, as well as how to make my dev and prod database handling more homogenous. And I can finally turn sendfile back on in my nginx configs without having to special case anything for dev!
Edit: Or is this not using any VirtualBox code at all?
Our solution was to create a custom fs watcher that would look for changes in the content of the files (it's only code, and for development, so speed doesn't matter much). I have been looking to replace this with something cleaner (at least with something like Dinghy and actual filesystem events), but Docker for Mac is likely the way we'll go.
It's been a much more friendly (read: plugin and play) than dinghy.
Native is better and I'm very excited for this.
I would love to see Docker come up with something that rivals Dinghy as I would prefer not to have to use a third party tool, but considering their reputation for stability and the comments being made about xhyve, I am happy to continue using (and gushing about) Dinghy for the foreseeable future.
Docker is a containerization standard that relies on various Linux capabilities to isolate application runtimes (or containers if you will). On Mac and Linux it used to be achieved by running a small Linux VM in VirtualBox, but looks like this release has brought xhyve on, which is supposed to have an even smaller foot print.
HTH.
The Docker for Mac and Windows beta does not use VirtualBox on either platform.
xhyve on Mac OS X is a very thin wrapper around Hypervisor.framework which itself is "entirely built into the OS".
Unfortunately, this means Docker only runs on Linux .. not even Linux...special Docker Kernel Linux (all the features they need are in the stock Kernel tree, but it's still a lot of modules). In Windows/Mac, you still need to run in a virtual machine.
Even with this update...you still need to run in a virtual machine. It's not actually running Docker natively. It can't, even on Mac which has a (not really) *NIX-sh base. You have to then use the docker0 network interface to connect to all your docker containers.
In Linux, you can just go to localhost. I _think_ FreeBSD has native Docker support with some custom kernel modules. I'm not sure...I've only looked at the Readme. I haven't tried it.
So even in Windows/Mac, all your containers do run in one VM (where as with traditional stuff you mentioned, you'd need a VM for each thing). Docker containers are meant to handle one application (that it runs as root within its container as the init process ... cause wtf?). With VMs, you'd typically want some type of configuration management (Puppet, Ansible, Chef, etc.) that sets up apps on each VM/server. With Docker, each app should be its own container and you link the containers together using things like Docker compose or running them on CoreOS or Mesos.
In my work with Docker, I'm not sure how I feel. LXC containers have had a lot of security issues. Right now, Docker doesn't have any blaring security holes and LXC has increased security quite a bit. CoreOS is pretty neat and I wouldn't use docker in production without it or another container manager (the docker command by itself still cannot prune unused images. After a while you get a shit ton of images that just waste space you're not using. CoreOS prunes these at regular intervals. A docker command to do this is still a Github issue. Writing one yourself with docker-py is horribly difficult because of image dependencies).
Oh and images. Docker uses images to build things up like building blocks. That's a whole thing I don't want to go into, but look it up. It's actually kind of interesting and allows for base image updates to fix security issues (although you still need to rebuild your containers against the new images ... I think...I haven't looked into that yet).
Docker is ... interesting. I find it lazy in some ways. I think it's better to build packages (rpms, debs). FPM makes this really easy now. Combine packages with a configuration management solution (haha..yea they all suck. Puppet, Ansible, CFEngine...they're different levels of horrible. Ansible so far has pissed me off the least) and you can have a pretty solid deployment system. In this sense, Docker does kinda make more sense than handling packages. You throw your containers on CoreOS/Mesos and use Consul for environment variables and you can have a pretty smooth system.
I dunno. I'm trying to actually like Docker. I've only made fun of it in the past, but now I work for a shop that uses it in production. O_o
:-P
Containers are run with whatever user you tell it to run as, the default is root because that's the only guarantee.
LXC is also something different. LXC is a set of userland tooling to interact with cgroups and namespaces (which docker used to exec out to). LXC != Linux containers (and indeed there isn't really such a thing as a container like there is a zone or a jail on Solaris and BSD respectively, it's made up) Also again, no custom kernel modules on BSD.
They also haven't solved the general case of keeping transient build data out of the final image either, but that's a broader problem that doesn't necessarily involve security concerns.
For now not a lot of people are concerned about either problem so it's not getting the attention it deserves. But they've been steadily peppered with inquiries about these issues for a year or two now and they still don't have an answer, which is concerning. I believe this is one of the reasons the CoreOS guys wandered off to do their own thing.
Fortunately for us and unfortunately for them, they have the design aesthetics of the Marquis de Sade, and until they start giving even half a thought to ergonomics, Docker is perfectly safe.
docker build --build-arg OAUTH_TOKEN=blah -t example .
The build-time environment variables were not designed to handle secrets.
By lack of other options, people are planning to use them for this.
To prevent giving the impression that they are suitable for secrets,
it's been decided to deliberately not encrypt those variables in the process.I think people aren't concerned about it because it doesn't make sense to try to put secrets into container images. Whatever you're using to deploy your Docker containers should make those secrets available to the appropriate instances at runtime. This is how Kubernetes handles secrets and provides them.
http://kubernetes.io/docs/user-guide/secrets/
(For example, what if you have two instances of a service and they need to have different SSL certs? Are you going to maintain two different containers that have different certs? Or would you have a generic container and mount the appropriate SSL cert as a volume at runtime?)
From my testing though, as long as you set the build-arg and consume it directly, it doesn't seem to persist. That said, it's super easy to fuck that up if the tool you consume it with then goes on to save the secret somewhere.
Thus it's no doubt best to use expiring tokens or keep your build seperate. Also don't use it to seed a runtime secret unless you treat, that'd force you to treat the image as a secret itself.
I think one of the things we're seeing is that Docker is opinionated, a number of powerful dev tools and frameworks are also opinionated, and us poor developers are stuck between a rock and a hard place when those opinions differ.
For instance I'm still not clear how you'd use the docker-compose 'scale' argument with nginx. Nginx needs to know what its upstreams are, and there's IIRC still an open issue about docker-compose renumbering links for no good reason, and some Docker employee offering up how that's a feature not a bug. I could punch him.
Single use auth tokens and temporary keys sure would fix quite a few things, to be certain, but those opinions keep coming in and messing up good plans :/
That said, as I said previously --build-args are dangerous, it's trivially easy to store then publish a secret, so it makes sense they weren't jumping for joy about implementing it. I'd say it is needed though, thus its now a thing.
And this is why I am stuck with a separate build and package phase, because I have to have that separation between the data available at build time and what ends up shipped, but even there I'm pretty sure I'm making mistakes, due to some of the design decisions Docker made thinking they were helping but actually made things worse.
For instance, there's no really solid mechanism for guaranteeing that none of your secret files end up in your docker image, because they decided that symlinks were forbidden. So I have to maintain a .dockerignore file and I can never really be sure from one build to the next that I haven't screwed it up somehow. Which I will, sooner or later.
I'm always one bad merge away from having to revoke my signing keys. It's a backlash waiting to happen.
All that's keeping you from committing your credentials is a .gitignore file. They have the file, it works reliably, don't worry about it.
And it's possible a similar issue still exists in docker-compose but it's still open.
.gitignore keeps me from checking my files into git, but it doesn't keep me from publishing them in a docker image. So now I have a second way to screw up.
Are you saying that docker would include files that should have been excluded by .dockerignore? I'd be interested to learn more. Thanks in advance.
* # exclude everything
!README.md # include the README.
!run.sh # include the initiation script
https://docs.docker.com/engine/reference/builder/#dockerignore-file
https://golang.org/pkg/path/filepath/#MatchThere are a couple of frameworks where all of the production files end up in, for instance /dist and one other directory. Rather than having to constantly blacklist everything you just say "ignore everything except X and Y"
I think but am not 100% certain this is the issue I was thinking of, but it seems the most likely, and it was just fixed in 1.10: https://github.com/docker/docker/issues/17911
Some day I'm sure .dockerignore will be solid, but my confidence level isn't high enough yet (it's getting there) to base my trust on.
My point was that there are other ways that directory structures and what is visible to COPY could have played out where vigilance is less of a problem. It's usually immediately obvious if a file you actually needed is missing from a build, but less obvious that a file that you categorically did NOT want to be there is absent.
Because the system runs in one of those scenarios and dies conspicuously in the other.
Nope, we've been using our own implementation of a container runtime for 2 years (libcontainer). LXC is not supported anymore and it was always a hacky execdriver.
> In Linux, these aren't VMs and are light weight user-land separations that use things like cgroups and lots of really special kernel modules for security.
They're kernel-space separations since the kernel understands namespaces (though it doesn't understand the concept of a container and some things aren't namespacrd).
> Unfortunately, this means Docker only runs on Linux .. not even Linux...special Docker Kernel Linux (all the features they need are in the stock Kernel tree, but it's still a lot of modules).
Almost all modern distros have support for all of the modules required to run Docker.
> In Linux, you can just go to localhost. I _think_ FreeBSD has native Docker support with some custom kernel modules. I'm not sure...I've only looked at the Readme. I haven't tried it.
FreeBSD is not supported as a daemon.
> So even in Windows/Mac, all your containers do run in one VM (where as with traditional stuff you mentioned, you'd need a VM for each thing).
Actually, recent versions of Docker can run as a daemon on Windows using some proprietary features I don't care about.
> Docker containers are meant to handle one application (that it runs as root within its container as the init process ... cause wtf?).
All machines have a single process running as root as the init. You can run a proper init inside your container (in fact it's recommended), and run many processes inside the same container. It's discouraged for scalability reasons to stuff your database and front-end in the same container because then it's hard to spin up more than one front-end connected to the same backend.
> In my work with Docker, I'm not sure how I feel. LXC containers have had a lot of security issues. Right now, Docker doesn't have any blaring security holes and LXC has increased security quite a bit.
Again: Docker doesn't and hasn't used LXC for quite a while. In addition, Docker has default selinux, seccomp and apparmour profiles that increase the security (seccomp allows us to disable syscalls that arent namespaced). There is a concern on the kernel side that they don't appear to care about going the Zones or Jails route: actually making the kernel aware about containers so that it can properly namespace things.
> After a while you get a shit ton of images that just waste space you're not using. CoreOS prunes these at regular intervals. A docker command to do this is still a Github issue. Writing one yourself with docker-py is horribly difficult because of image dependencies).
ahem % docker images | awk '/^<none>/ { print $3 }' | xargs docker rmi
Sure, it's not a single command but it isn't impossible to do and doesn't require docker-py. Besides, you should be using engine-api.
> Oh and images. Docker uses images to build things up like building blocks. That's a whole thing I don't want to go into, but look it up. It's actually kind of interesting and allows for base image updates to fix security issues (although you still need to rebuild your containers against the new images ... I think...I haven't looked into that yet).
There's also tools like zypper-docker to allow for hot-patching of images.
If you take free software and never consider paying its developer for making it, despite them providing you freedom, choice, and a degree of trust in the software you can not have with proprietary code, then you are the kind of person to blame for why proprietary software is so rampant today.
For example, I donate $200 to the Document Foundation every year to match the cost of an annual subscription to Office 365 plus a 33% bonus for respecting my freedom.
You mean freely sharing execution? The source is not the idea.
We will open-source all the components individually, to make them easier to reuse elsewhere. That requires work to do properly.
Lastly, Docker for Mac and Docker for Windows will be free.
I've wrote a blog post about this topic a few months ago, check it out if you're interested in a simpler way of building development environments: https://www.mpscholten.de/docker/2016/01/27/you-are-most-lik...
I'm a Docker n00b, still don't know what it can do exactly. Can Docker replace Virtualbox? I guess only for Linux apps, and suppose it won't provide a GUI, won't run Windows to use Photoshop?!
Previously in order to run Linux containers on a Mac, you needed to install VirtualBox and have an embedded Linux virtual machine that would run the Docker containers from the Mac CLI. There would be a network endpoint on your Mac that pointed at the Linux VM, and the two worlds are quite separate.
Docker for Mac is a native MacOS X application that embeds a hypervisor (based on xhyve), a Linux distribution and filesystem and network sharing that is much more Mac native. You just drag-and-drop the Mac application to /Applications, run it, and the Docker CLI just works. The filesystem sharing maps OSX volumes seamlessly into the Linux container and remaps MacOS X UIDs into Linux ones (no more permissions problems), and the networking publishes ports to either `docker.local` or `localhost` depending on the configuration.
A lot of this only became possible in recent versions of OSX thanks to the Hypervisor.framework that has been bundled, and the hard work of mist64 who released xhyve (in turn based on bhyve in FreeBSD) that uses it. Most of the processes do not need root access and run as the user. We've also used some unikernel libaries from MirageOS to provide the filesystem and networking "semantic translation" layers between OSX and Linux. Inside the application is also the latest greatest Docker engine, and autoupdates to make it easy to keep uptodate.
Although the app only runs Linux containers at present, the Docker engine is gaining support for non-Linux containers, so expect to see updates in this space. This first beta release aims to make the use of Linux containers as happy as possible on Windows and MacOS X, so please reports any bugs or feedback to us so we can sort that out first though :)
The bug above has been reported to Apple and they've reportedly fixed it in the latest 10.11.4 seeds, but we've put in a workaround that detects ACPI sleep events and freezes vCPUs just before going into hibernate mode. None of the beta testers have reported any sleep crashes using Docker for Mac recently, so if you do see anything of this nature please let us know.
Perfect. We had to ditch Kitematic on OS X due to the lack of port forwarding, since we couldn't get OAuth redirects to work when developing locally.
I think they forgot about Linux :)
Edit: Oh, and I don't understand the comment about printing. Cups works.
It may also work for your particular device scenario, but there are thousands of scenarios (networks, devices, etc) in which its functionality may be limited or practically non-existent.
CUPS and Amahi work great for me. I've had more pain setting up printers on Windows and OS X (the latter started sending print jobs on each probe).
> run one of the bajillion pieces of useful software that aren't available for Linux.
Examples? I can't think of any software that I need that isn't available for GNU/Linux.
Or when you want a distro like arch on a laptop...
> Many of the OS-level integration innovations will be open sourced to the Docker community when these products are made generally available later this year.
Does this mean it is closed right now?
https://news.ycombinator.com/item?id=11352594
I imagine a lot of this work will also be useful for developers wanting to test all sorts of unikernels on their Mac and Windows machines.
Please do check it out and suggest some particular benchmarks that are important to you -- we're busy building up a performance benchmark suite atm.
The importing thing is that WTP doesn't manage the server lifecycle.
Arun Gupta wrote many excellent posts on how to use Docker to build Java apps: http://blog.arungupta.me/docker-tooling-eclipse-video/ on Eclipse tooling for Docker, http://blog.arungupta.me/deploy-wildfly-docker-eclipse/using Wildfly
https://github.com/chanezon/docker-tips/tree/master/orchestr... is an example leveraging Docker Compose and swarm for a Spring Boot application.
I hope these helps get you started using containers for development.
At the moment VMWare Workstation users will be a bit left out as Windows doesn't like having two hypervisors installed on the same system...
We're using Vagrant for development environments, and as the number of microservices grows - the feasibility of running the production stack locally decreases. I'd be interested in learning how to spin up five to ten docker services locally on OSX for service-oriented architecture.
This product from Docker has strong potential.
The django quickstart guide is a good starting point for wrapping your head around it, https://docs.docker.com/compose/django/
Would you mind trying again but allowing Marketo?
I work for a web agency and currently, our engineers use customized Vagrant boxes for each of the projects that they work on. But that workflow doesn't scale and it's difficult to maintain a base box and all of the per project derivatives. This is why Docker seems like a no-brainer for us.
However, it became very clear that we would have to implement our own tooling to make a similar environment. Things like resolving friendly domain names (project-foo.local or project-bar.local) and adding in a reverse proxy to have multiple projects use port 80.
Docker for Mac looks like it will solve at least the DNS issue.
Can't wait to try it out.
edit: words
If I were a 12 year old girl I would be "squee-ing" right now. Ok, I'm lying - I'm a 40 year old man actively Squee-ing over this.
:)
It really plays nicely into my "weekend-project" plans to write a fully containerized architecture based in dotnet-core.
Nathan LaFreniere (the author of dlite) is awesome, and we've been exchanging tips and tricks and areas where we can collaborate. He knew exactly where to press to find bugs in our earlier betas...
At the moment I use dlite. The thing I love about it is that it's transparent. I hope that the new Mac app has an option or mode to be like that too (start on system boot, doesn't create a new desktop window/gui, SSH from terminal would be enough for me).
Something analogous to MacVim's -v flag; by default "mvim" opens a new app with its own window, but "mvim -v" starts Vim inside current terminal. Not a great analogy, sorry about that.
Thanks.
Docker Toolbox just brings back too many nightmares from Adobe's awful Updater apps.
I don't think Docker can do anything about case sensitivity, but with this new release will permissions differences be handled better?
Damnit man, we just want your beta, not your excuses :P
So you can only use it for development. And it is close sourced. hmmm...
Is there a user/password length limit? (I used a 30char user/password. 1password FTW).
I know it's a small thing, but it's kind of a pain sometimes.
also, Oracle.
"Many of the OS-level integration innovations will be open sourced to the Docker community when these products are made generally available later this year."
If they do decide to block browsers from accessing localhost, will that impact docker?
which is usually like, what most people would have, and would consider the standard windows install. Windows Home is... very nutered to say the least
Some of the missing features in Home are what I'd describe as "immoral." In that they aren't just luxuries, they're important parts of the OS or security features e.g.
- Group Policy Editor: This is the primary place to modify hundreds of local computer settings. They could have left out Domain Join, and kept the local Group Policy Editor.
- Start Screen Control with Group Policy: Adds more group policy options to modify the start screen/menu look/feel.
- Enterprise Mode Internet Explorer: Name notwithstanding, this allows people to use legacy webapps with modern IE.
- AppLocker: Security feature (isn't even in pro incidentally!). I'd turn it on.
- Bitlocker: Full drive encryption (with different decryption options).
- Credential Guard: Not used to protected non-domain credentials.
- Trusted Boot: Because home users don't get rootkits?
Windows 10 Home is categorically less secure than Windows 10 Pro, which is in turn less secure than Windows 10 Enterprise. Features like AppLocker, Credential Guard, Trusted Boot, are features that all versions of Windows could benefit from, and Bitlocker should be available and on by default.
When you have a "security" category in the feature list and are differentiating different versions of the OS then you really have to ask yourself how high you prioritise security in general.
From the OP:
"The Docker engine is running in an Alpine Linux distribution on top of an xhyve Virtual Machine on Mac OS X or on a Hyper-V VM on Windows"
This is achieved by using the native virtualisation support on OSX / Windows instead of Virtual Box and working closely with Apple and Microsoft.