Announcing Docker ToolBox | Hacker News Reader

Announcing Docker ToolBox

blog.docker.com

110 pointsanand-s70 comments

Nice one. However, VirtualBox's shared-folder support is so god-awfully slow, that here at work we all swapped out the regular boot2docker setup for boot2docker-xhyve[0] which is so much faster at nearly everything that it's not even a comparison.

Now, with ailispaw's dockeroot-xhyve you can even get a VM root image that can be grown on the fly, so you never run out of space on your VM when building large Docker images.

In addition, I'd never recommend Docker Compose to web developers that rely on databases, as quirks with it have caused many a volume-only container to be destroyed and rebuilt, thus defeating the purpose.

[0] https://github.com/ailispaw/boot2docker-xhyve

Interesting, thanks for sharing! I agree that the shared folder situation is catastrophic. Doing anything significant with the mounted folders, like doing builds or simply watching for changes is unbearably slow. What I generally do is have a 'rsync' container and attach its volume (--volumes-from) to all containers that depend on the folders. Then, I watch on local file changes and rsync to the container. Builds etc. are all done within containers and this seems to work great!

Docker under OSX is a real pain to solve ASAP.

I've set Docker on my Linux machine and it's working like a charm. Then I had to do the same for my coworkers running OSX, the vbox shared folders are definitely unusable.

Now it's been few days that I'm trying to find the best way to have a two-way sync in order to sync back changes from docker to the OSX folder (eg. when you upload a file and it's saved in the /public folder, otherwise it'll be lost)

So far I've used docker-osx-dev[0] for 1 way sync (with rsync) and it's working nicely.. they also plan to add the support to unison for a 2way sync

I've also found docker-unison[1] but I haven't found a way to have it working correctly

I'd like to try with boot2docker-xhyve, but it only runs on Yosemite and later

[0] https://github.com/brikis98/docker-osx-dev

[1] https://github.com/leighmcculloch/docker-unison

There's https://github.com/codekitchen/dinghy if you want to mount the Mac host volumes inside boot2docker via NFS. We have a Vagrantfile that does this, but dinghy packages it all up nicely.

At the end of the day though, Docker as a development environment is really bad on OS X until a VM vendor fixes file share performance. I really wish Docker would make it a priority to have decent Mac support.

VMware Fusion has fine performance on shared folders. I use it all the time with docker-machine.

I use docker for development on Mavericks using vagrant + coreos as the host VM and things works nicely [0]. NFS shared folders performance is fine and the configuration is trivial.

[0] https://github.com/coreos/coreos-vagrant

Shared folder performance has been a known issue for as long as I can remember, with the recommended solution being to use NFS.. Is this not possible on OSX?

Try Hyper on Mac: https://hyper.sh/blog/post/2015/07/30/running-containers-fro...

Let me chime in and say that getting rid of VirtualBox and moving to xhyve would be amazing. I was really sad when this PR was closed:

https://github.com/docker/machine/pull/1358#issuecomment-126...

It will happen :) Xhyve is pretty new (and only works on pretty recent OSX installs), but we are also feeling the pain of the default vbox install and really want to fix it.

That's great news, Solomon - thanks :)

Has anyone tried VMWare fusion to run a Docker host VM?

Virtualbox shared folders are unusable for my purpose (compiling software on Linux).

Yes, and I think its much worse in terms of reliability. When I save a file (say a Gemfile), sometimes only half of it shows up inside the container.

Anybody know what would cause VMWare Fusion (or any VM for that matter) to behave this way?

I run into this. Files are only partially synced.

You end up with broken code. If you are lucky it has a syntax error. If you are not lucky you get mysterious bugs.

The work around is to go back and try re-editing the file to retrigger a sync.

There are lots recommendations to try Fusion over Vbox but I think this issue makes it a wash.

This is a known bug, we're addressing it with a new version of open-vm-tools, there's a thread going on over at github[1] to track this.

[1] https://github.com/docker/machine/issues/1460

> Has anyone tried VMWare fusion to run a Docker host VM?

If you're considering Fusion I'd suggest taking a look at AppCatalyst[1] instead (N.B. I work at VMware, but not on AppCatalyst).

[1] http://blogs.vmware.com/cloudnative/vmware-appcatalyst/

looks great, but doesn't have a docker-machine driver yet

https://github.com/docker/machine/pull/1401

My branch is in working order, you can build and use it with appcatalyst: https://github.com/frapposelli/machine/tree/appcatalyst-driv...

Is there a good way to connect with the developers working on appcatalyst? It seems really promising but it also seems like there are fundamental parts of the developer workflow that are either not implemented yet or poorly documented. Having put up with the hassles of VirtualBox when using boot2docker, a free/low-cost VMWare option is a really great development, but it doesn't seem entirely usable yet and needs some of the layer that Boot2Docker gives you on top of VirtualBox before it will be possible to switch.

I'm part of the dev team that is responsible for appcatalyst, feel free to reach out: fabio at vmware dot com

Yes, most of us where I work do this. Fusion runs about 3x faster for our system, so using virtualbox would seriously impact development time; most of us use VMware shared folders and SSH into the VM, so it feels pretty seamless. I haven't had any of the problems that others are saying they have.

It will be available in docker-machine v0.5.0 as a plugin.

What should I use in place of docker compose?

I strongly recommend you to use azk [0]. It's an alternative to docker compose but instead of being a container manager, it provides the whole environment orchestration. When compared to docker compose, azk has several advantages, such as:

  - Its manifest file is more verbose and has a greater toolset [1];

  - You can persist content from inside the container way better than simply mounting volumes using docker compose [2];

  - Fow now, it relies on VirtualBox (usign debian2docker) but it has a built-in function [3] to sync folders into the container using rsync (significantly increasing performance for file sharing). You can still use VirtualBox Shared Folders if you want to;

  - Its built-in DNS server makes you don't worry about port mapping and docker IP issues. You can access your local app running with azk by simply accessing a custom address, like: my-app.dev.azk.io.

[0] http://azk.io

[1] http://docs.azk.io/en/azkfilejs/README.html

[2] http://docs.azk.io/en/reference/azkfilejs/mounts.html#persis...

[3] http://docs.azk.io/en/reference/azkfilejs/mounts.html#sync

> In addition, I'd never recommend Docker Compose to web developers that rely on databases, as quirks with it have caused many a volume-only container to be destroyed and rebuilt, thus defeating the purpose.

Can you point me in the direction of a bug report or a more concrete example? I'd like to make sure we follow up.

Some of it is insurmountable without having some way of marking a container in the docker-compose.yml as "never destroy this!". It's a UX thing more than a bug, although there was two cases where "docker-compose up" destroyed and rebuilt a volume-only container that was in the docker-compose.yml -- unfortunately it was difficult to replicate so we didn't send a bug report through. The big thing is that using it on OSX with boot2docker means the cache can get confused, and so to get it to pick up config file changes (say, changes to an nginx config that are brought into the container) or some Dockerfile changes requires a "docker-compose rm && docker-compose build --no-cache", which of course will delete the volume-only container unless you're aware enough of the need to name the container you want to rebuild; as I said, it's more a UX issue than a bug, in my opinion.

In terms of fixes, ideally I'd like to see a way of marking a container as "volume-only" -- this would then mean a "docker-compose rm" would not remove it, and "docker-compose up" would never rebuild it.

Great release!

I bought a new MacBook 2 days ago, and setting it up has never been easier.

Almost all of my development takes place in a Docker environment now, and the Docker ToolBox makes installing that a breeze.

Less XCode, Homebrew, rubygems, npm. More Docker.

This looks cool, although for the Windows install I'd have hoped that there would be an option to integrate Hyper-V with it instead of Virtualbox, seeing as Hyper-V is available on all Windows 8.1/10 pro/enterprise systems, and having multiple virtualization platforms active on a system is generally a bad idea...

There is a hyper-v driver for docker-machine. You can use it like so:

    docker-machine create -d hyper-v vmname

nice, thanks!

Hi all. We know the experience of using Docker on OSX is not perfect today. The default virtualbox configuration for file sharing is slow and buggy. You need to use 3 different CLIs (`docker`, `docker-machine`, `docker-compose`) which are not well integrated. 3d-party clients need to reimplement everything themselves. The standard clients are not configurable or customizable enough.

We plan on solving all of this. There is a new team at Docker focused entirely on solving "Developer Experience" problems. This initial release of Toolbox is a starting point: a convenience packaging of existing tools. But we are going to update it with more and more improvements.

In short: we are going to work hard to make the experience of using Docker to develop on Mac OS X (and Windows) much, much better.

Hi Solomon,

It is great to hear you guys are working on that. I find Hyper is pretty interesting to run Docker on Mac https://hyper.sh/blog/post/2015/07/30/running-containers-fro....

I think Hyper is interesting as a backend for Docker in production use cases where you need to extra isolation of VMs... But in development in doesn't make much sense to run each container in a separate VM. So I'm not convinced we need this extra layer of abstraction.

I really, really, don't understand the docker fixation, but maybe I'm missing something.

The most important function of docker, IMHO, is that it standardizes the creation of a VM using Virtualbox across platforms.

Unless Amazon AWS, DigitalOcean, GoogleCompute, etc. allow a docker image (aka a VirtualBox image) to run directly on their base virtualization platform (without me putting together an AMI or whatever on their platform), this just adds another layer of virtualization that I can do without. Do they do this already?

If and when they do that, docker would be tremendously useful because I wouldn't need to rebuild my image for that particular platform (AWS/DO/GC).

And if they do that already, then why aren't they just accepting existing popular VM images (VMWare/VirtualBox)?

AWS supports deploying Docker images on both Elastic Beanstalk and their new Elastic Container Service. While it doesn't completely remove the need to worry about VMs, it does abstract them away somewhat (ECS much more so).

Does the docker image run directly on their base virtualization platform or does it run on top of another OS (that I install) that runs on their virtualization platform? I have an OS in my docker image already, so..

But you don't have an OS in your docker image really. There's no kernel, no init or system services running if you don't do that yourself. A running docker container is just a process on the host OS with it's own environment.

ECS abstracts this away, and from your perspective, containers are running on their virtualization platform.

The "OS" part of a Docker image is really just a minimal environment that "looks and feels" like Ubuntu, CentOS, etc. You're not actually virtualizing a full OS. Running a Docker image is essentially equivalent to running a process (whatever process is defined by your image, i.e. Apache, MySQL, etc.).

Docker Machine alone knocks out the entire list of hosts you laid out: https://docs.docker.com/machine/

For more sophisticated scheduling needs there is http://www.replicated.com/, https://cloud.google.com/container-engine/, and https://aws.amazon.com/ecs/

Related to what you say about "don't VM's solve this problem?" is a project called https://hyper.sh. They're coming at it from the angle that if VM's can boot really fast, containers are a moot point.

So docker machine will create an image for that particular host (AWS/DO/GC...) and create a docker image on TOP of that, correct? It solves the problem of me configuring multiple hosts, but there's still an extra OS layer in there, isn't it?

Docker machine creates and manages VMs, then you can run any number of containers on those VMs.

You're misunderstanding what Docker does.

The most important function of Docker is dependency management (allowing you to pack your own runtime with your apps with consummate ease), followed by standardising image distribution and deployment across environments.

A Docker image is not a VM image - it doesn't have a kernel, for starters, and is not designed to be booted in the usual sense (i.e., kernel+init+scripts) - it just has a Linux userland (often quite restricted) and will run _a single process_ by design[1].

Also, AWS and Google Cloud already allow you to run Docker images (on EC2 Container Service and Kubernetes).

Cloud providers won't accept existing VM images in the sense that large-scale hypervisor platforms usually use different formats and they don't need the hassle involved in building and supporting VM conversion services.

It's pointless to have those as a service when anyone can grab Packer or a similar tool and convert a .vbox/.vmdk to an AWS AMI or an Azure Hyper-V image - and support the resulting images themselves (which is another reason why providers make a clear distinction between certified/tested images and third-party ones).

[1]: of course you can launch supervisord and a bunch of stuff underneath it, but if you're doing that, you're doing it wrong.

Ok, this is starting to clear things up for me.

When you say that it runs a single process by design, does it mean that if my application creates a fork(), the image will be unable to handle scheduling of the additional process?

Let's say I have an application that talks to MySQL. What is the right way in docker? Should I have two separate docker images, one running the application and the other running MySQL, with them talking to each other across the host OS? Or can they be configured in a single docker image?

> Should I have two separate docker images, one running the application and the other running MySQL, with them talking to each other across the host OS?

Yes.

> Or can they be configured in a single docker image?

They can, but it's an anti-pattern and there are hurdles to overcome if you try managing processes using an init system.

Thanks. Any idea about this?

> ...if my application creates a fork(), will the container be unable to handle scheduling of the additional process?

Forking works fine.

Hmm... if there's no kernel, who manages the process scheduling?

It uses the host OS kernel, which is why its limited to a subset of Linux host machines.

Honestly Docker is the epitome of the incorrect understanding of "DevOps".

There is a kernel, Docker uses the host's kernel it runs on.

People differ on this point. Discourse, for example, ships a single Docker container which runs the Rails-based webapp, a database server (PostgreSQL), and a key-value store (Redis). They have been using this same setup in production for quite a while now.

So how do they run multiple application servers to scale horizontally? Or a cluster of Redis servers?

I think you are confusing Docker with Vagrant. Vagrant provisions VM, Docker is about containers (which may or may not run on a VM)

I gave up setting up my mac for Docker. It's just way easier to set up another dev machine with Linux since it can be used as dev machine these days.

Just use GNU/Linux already!

It's true. Docker Just Works™ on GNU/Linux. I'm a Linux guy myself for anything in development. However, unless I'm mistaken, part of the goal of Docker is to provide identical environments to an engineering team and production.

I always advocate for Linux use (in addition to principles, most things are easier as well), but everyone has their preferred environment.

You can stand up Joyent's Triton[1] service locally for development with Docker. The hardware is very affordable, and I think that they have a very underrated product on their hands.

[1] https://www.joyent.com/blog/spin-up-a-docker-dev-test-enviro...

Anyone have any idea how to configure this with a proxy in OSX? It seems to be ignoring the HTTP_PROXY env variable

Or, just run docker directly on Linux?

I appreciate the Toolbox suite as I use most of them. But I am concerned though not surprised how this make the "batteries included but removable" motto more insignificant as it becomes more unlikely that people will use alternatives.

I use Vagrant+Docker for development, no problems to install. First, I tried to use boot2docker but I didn't like it. Vagrant has a Docker provisioner or there's a plugin for Vagrant for Docker Compose.

Does anyone have any idea how to get the Docker Quickstart Terminal app to use iterm2/zsh instead of Terminal/bsh?

I didn't use it, and ended up following the instructions here: https://docs.docker.com/installation/mac/#from-your-shell

If you look at the contents of the .app, you can edit main.scpt in AppleScript Editor, but it's really just doing two things, creating the VM if necessary:

docker-machine create -d virtualbox --virtualbox-memory 2048 default

and populating environment variables for docker:

eval $(docker-machine env default)

Passing --shell zsh to docker-machine env doesn't seem to change its output. Does zsh understand commands like the following?

export DOCKER_MACHINE_NAME="default"

Yep, `docker-machine env` should work fine in zsh (I am a daily zsh and docker-machine user).

Sadly no support for Parallels.

This is probably why

https://www.google.com/trends/explore#q=parallels%20mac

The docker team completely ignored the great work the parallels team put into creating a parallels driver instead telling them to wait for the plugin architecture to be finished.

Reference: https://github.com/docker/machine/pull/939

That is a bit of a bummer, but do I understand correctly that the driver only supports a yet-to-be-released version of Parallels?

It was a tough decision, and "completely ignored" is an unfair way to put it.

The Machine maintainers are overwhelmed by pull requests adding new drivers. They are worried about maintenance load, and its impact on the quality of drivers over time. So instead of making promises they can't deliver on, they're investing the time on a plugin architecture, so that everyone can create and add their own drivers without depending on the bottleneck that is the core maintainers.

boot2docker was working fine... Docker Toolbox is broken

Reply on news.ycombinator.com