Docker for Mac Beta Review
medium.com
medium.com
A few issues I've seen:
1. I cannot believe they are using `docker.local`. This hostname will cause nothing but trouble for years to come. DON'T USE `.local`! Apple has decided that `.local` belongs to Bonjour, and due to a longstanding bug with their IPv6 integration, you can expect to see a 5-10s random delay in your applications as Bonjour searches your local network to try to resolve `docker.local`. Yeah, you put it in your `/etc/hosts`? Doesn't matter. Still screws up. Use `docker.dev` or `local.docker`. [http://superuser.com/questions/370559/10-second-delay-for-lo...]
2. -beta8 is screwed up. It won't bind to its local ip anymore. The only option is to port forward from localhost. Unfortunately, Docker isn't offering a download of beta7. Thankfully, I still had the DMG around. 3. The polish is still lacking. Most menu bar items ask you to open up something else. 4. Why "Docker for Mac"? Couldn't the team think of a less confusing name? Now I have "Docker" running "docker".
Otherwise - great projects, and again, much credit to @nlf for `dlite`. If you're not part of the beta, check out dlite (https://github.com/nlf/dlite). It's at least as good as Docker for Mac.
> The implementation of both approaches on the same network can be problematic, however, so resolving such names via “unicast” DNS servers has fallen into disfavor as computers, printers and other devices supporting zero-configuration networking (zeroconf) have become increasingly common.
Which seems to confirm what the original poster wrote - it sounds like a bad idea using it on OSX where it collides with bonjour.
Why should everyone else change their ways because apple made an error?
http://www.iana.org/assignments/special-use-domain-names/spe...
Any DNS query for a name ending with ".local." MUST be sent to the
mDNS IPv4 link-local multicast address 224.0.0.251 (or its IPv6
equivalent FF02::FB)There are some reserved suffixes, including .localhost and .test: https://iyware.com/dont-use-dev-for-development/.
Not sure if that is important for the use-case at hand, but it was the reason why I stopped using .localhost in my LAN (in favor of just using plain hostnames, e.g. “x1”).
We are indeed moving away from `docker.local` in Docker for Mac. There have actually been two networking modes in there since the early betas: the first one uses the OSX vmnet framework to give your container a bridged DHCP lease ('nat' mode), and the second one dynamically translates Linux container traffic into OSX socket calls ('hostnet' or VPN compatibility mode).
Try to give hostnet mode a try by selecting "VPN compatibility" from the UI. This will bind containers to `localhost` on your Mac instead of `docker.local` and also let you publish your ports to the external network. One of our design goals has been to run Docker for Mac as sandboxed as possible, and so we cannot just modify the /etc/resolv.conf to introduce new system domains such as ".dev".
We've been iterating on the networking modes in the early betas to get this right, so beta9 should hopefully strike a good balance with its defaults. It's also why we've been holding a private beta, so that we can make these kinds of changes without disrupting huge numbers of users' workflows. Your feedback as we figure it out is very much appreciated!
pinata set native/port-forwarding true
In beta 9 using localhost will be the default -- as avsm says we've been iterating on the network configuration trying to find the most compatible / least surprising defaults. Hopefully after beta 9 it will be stable on "localhost".
Pretty much every developer's workflow is already heavily disrupted by not having access to beta. I've spent days trying to get my filesystem notifications up with dlite and Dinghy (succeeded with the latter). So whatever you guys do, it's still better than what's currently available.
Keep up the awesome work!
Maybe it's time for an RFC that creates a guaranteed-not-to-be-public TLD, kinda like test.com? Until then, docker.<completely-NSFW-slur> might be your best bet, actually.
Docker for Mac/Windows, once released, will nuke the ick factor on those platforms from orbit, which can only lead to even more adoption.
1) If you have a container per data object, doesn't that mean you also have to start a process every time a user opens a document? So forget about doing any computations in the setup. Even just things like using regex patterns in python (that need to be compiled once) or using anything with a VM (that needs to be started) you'd have to give up. Usecase seems to be extremely limited, but maybe I'm not getting something right here.
2) How do you handle indexes, views and collections over a large set of data objects?
Not at all! The app market is full of apps that were not originally written for Sandstorm:
Examples: Wekan, Etherpad, Rocket.Chat, EtherCalc, draw.io, Gogs, Dillinger, NodeBB, EtherDraw, ...
It turns out that converting a web app to Sandstorm is mostly deleting code. You delete your user management, your collection management, your access control, etc. What you have left is the essence of your app -- the UX for manipulating your core data model, of which you now only need to worry about one instance.
> doesn't that mean you also have to start a process every time a user opens a document?
Most apps we've encountered only take a couple seconds to start. But we're working on a trick where we snapshot the process after startup and start each grain from the snapshot, thus essentially optimizing away any startup-time slowness.
> How do you handle indexes, views and collections over a large set of data objects?
Sandstorm is (currently) designed for productivity apps, not for "big data" processing. The data within a single grain is usually small. That said, you can run whatever database you want inside the grain.
(I'm the tech lead of Sandstorm.)
Actually I can't think of any app that matches the abstract problem you describe. Can you give a specific example? Usually the answer to these problems becomes much clearer in the context of a specific app.
So I guess my question is: How do you expect people to use your fine grained model with databases? If the answer is "not at all" then I find the scope too limiting. If the answer is "1 grain == 1 db" then I find the claim that you are solving difficult permission problems to be false.
Note: I don't want to be too critical here, I'm just trying to pick holes in your claims of scope so I can categorise the power of sandstorm and how far it could be useful for things I'd like to build.
Wekan is a Trello clone that uses MongoDB for storage. On Sandstorm, each board lives in a grain, so there ends up being one MongoDB per board. This works fine. The only thing stand-alone Wekan ever did that queried multiple boards at once is display the user's list of all boards. On Sandstorm, displaying the user's grain list is Sandstorm's job, not Wekan's -- and indeed, usually the user is more interested in seeing the list of all their grains rather than just the Wekan boards, so delegating this to Sandstorm is a UX win.
If that is not the kind of example you have in mind, then you really need to give a specific example.
This is very interesting. I've been looking for something like this since 2007 for optimizing the startup time of some apps. However I couldn't find any suitable technologies for this purpose; VM memory snapshotting is heavyweight and is slower than starting an app from scratch, OS-level tools like cryopid can't even be called alpha-level. What kind of snapshot technology do you intend to use, and how confident are you that it will work well?
I'm going to guess it'll get better in time. It would be nice to get some insight into just what is burning CPU cycles. The experience besides that was really top notch IMO.
The early betas focussed on feature completeness rather than performance for filesystem sharing. In particular, we have implemented a new "osxfs" that implements bidirectional translation between Linux and OSX filesystems, including inotify/FSEvents and uid/guid mapping between the host and the container. Getting the semantics right took a while, and all the recent betas have been steadily gaining in performance as we implement more optimisations in the data paths.
If you do spot any pathological "spinning cases" where a particular container operation appears to spiking the CPU more than it should be in, we'd like to know about it so we can fix it. Reproducible Dockerfiles on the Hub are particularly appreciated so that we can add them to the regression tests.
Are we going to see these changes rolled back upstream in xhyve?
Our approach is to focus on functionality and correctness first, and then improve performance over time.
We're building up a suite of performance benchmarks to help us track progress-- are there particular benchmarks that you would recommend we add? I'll certainly add "CPU load while idling" to the list.
Just my two cents: I think that the interface and features are excellent and a joy to use, but performance was a show-stopper that forced me to quit using the Beta for Rails-application development:
- a simple request that took ~1/3 of a second using VirtualBox and Docker Machine took six seconds on Docker for Mac - a more complex request went from one second to twelve seconds
I didn't have a chance to dig into it, but I would guess that it has something to do w/ osxfs and the many files that Rails loads, as it reminded me of the difference between using sharing files via VirtualBox's own file sharing vs. NFS, the latter being a significant improvement.
Definitely check out the beta forums as most of the issues I've found have already been reported with workarounds.
I don't see big companies using something this hackish for containers that are running on servers anyway. For working on the desktop this might come in handy for devs, but honestly I think MS should focus their energy on something else.
But you'd actually need many different kinds of "Windows container", since Windows actually has an abundance of kernel-exposed runtimes: the DOS VMM, Win16 with cooperative threading, Win32 with COM, WinNT, WinRT, the POSIX subsystem...
You could certainly write a particular container runtime to allow a specific type of app (e.g. WinRT apps) to run, and that might be enough to enable developers going forward to target both Windows and Linux hosts for their Windows apps. But that would hardly be Windows, in the sense of being able to have your app launch arbitrary other "Windows" processes in the same container the way that Docker apps do with arbitrary Linux processes.
Having all the machinery to simulate all the vagueries that have changed in the Windows OS core over time, such that one container could contain any and all Windows processes running together, would be a much harder challenge. I don't know what the combined surface area of all the runtimes the Windows kernel exposes looks like, but I can't imagine it'd be something even MS could re-implement as a Linux-kernel translation layer easily (especially considering all the compatibility shims each layer provides to make specific apps work, that would have to be carried forward into the translation layer.)
Running Linux on Windows without a VM would be a godsend. And no, Cygwin doesn't count.
It's actually on a windows subsystem that can run linux ELF64 executables natively on windows:
https://blogs.msdn.microsoft.com/wsl/2016/04/22/windows-subs...
Native Docker support in Windows will be available in the next Windows server release, and is already available in the Windows technical preview bits (since TP4).
The only problem I had with docker was that I did not use to support shared volumes that are outside the home folder on Mac (I think they changed that now, but I'm not sure).
Which actually makes me wonder how they're managing memory for the VM hosting Docker here. Are they specifying a set fixed allocation? Is memory usage configurable somewhere?
With the beta, all of that is taken care for you with a couple of settings, and it's just much simpler to get up and running.
Except on Mac, where it sometimes doesn't. Have you not run into permissions issues? Or having a file-watcher process running in the container not being triggered by changes from the host? There are a tonne of quirks that came with boot2docker. It was worth it, of course, but they were still there; and this app is specifically aiming to address them.
I also recommend running "pinata set network nat".
There is still a tiny VM running. This one happens to be the Native OS X Hypervisor Framework. From the docs:
> Hypervisor (Hypervisor.framework). The Hypervisor framework allows virtualization vendors to build virtualization solutions on top of OS X without needing to deploy third-party kernel extensions (KEXTs). Included is a lightweight hypervisor that enables virtualization of the host CPUs.
https://developer.apple.com/library/mac/releasenotes/MacOSX/...
I've had a great run with VirtualBox, between Vagrant and Docker Machine. But I can't lie, I won't miss its installer, uninstaller, OS X kernel extensions, questionable network file sharing, and more. Removing a big blob of software between me and my virtualization-ready CPU is progress.
Then Docker for Mac is the one-two punch. Simpler virtualization, extremely rich containerization.
At my El Capitano, the exact same setup in Docker Beta takes roughly ten times to do its thing than my more flexible vbox setup did. A java stack (Jenkins) starts in about 1.5 minutes, but with Docker Beta it takes 15 minutes or about!
So, my docker-machine setup lets me see my hosts with vbox, manage them with docker-machine, and get the NFS tweaked with docker-machine-nfs. boot2docker OS is nice and small and works.
So for me this is quite a contrast with the 'native' Alpine images based Beta. Which in my 5-hour stint with it did not show much way to overview or inspect it without getting new/more gear.
Anyhow, I really hope someone does a good overview of Docker for Windows beta, as well as the Ubuntu environment within Windows 10 now...Seems like OSX gets all of the dev love, so I'm wish and hoping for a really nice Windows overview. As I am currently having a hard time with both. Neither, as of right now, work well.
There is a ton of potential there. My biggest challenge is that the documentation hasn't quite caught up to all of the interesting stuff that is going on. I'd certainly welcome some more opinionated answers for how to develop on Docker. Specifically: how to not run apps as root, as almost all examples use root and permissions are annoying if you don't do so; how to use docker containers for both dev and prod; best practices for getting ssh key access into a container during the build phase.
But much of it Just Works at this point, I'm pretty confident that the best practices will catch up in time.
I install node, postgres, and redis natively and it all works fine. What benefits does docker provide to my workflow?
(Not to say Docker's immune from that; the sudden deprecation of docker-compose for docker-machine was a nasty surprise.)
I think you meant the deprecation of Boot2Docker?
Yes, we screwed up on the b2d->machine transition. Sorry for making that experience unnecessarily confusing. We've learned a lot from that mistake and are working hard to avoid repeating it.
With docker, you can make a dockerfile for your project and make it painless and consistent to run anywhere. You can also create a docker-compose file if you need other services like redis. It really is the holy-grail once it clicks.
When we developed an angular and java site, I set up vagrant to configure tomcat, node, java, and all of the plugins required to get tomcat and maven to be nice together. Did it once, and then everyone else with a unixy platform were able to not spend time on dealing with that. Now that the class is over, all of that is removed from my machine but I can always just crank it back up in the time it takes to install all of those dependencies.
If you develop everything using Docker, running the containers on another (Linux) computer is much easier as everything is already prepared and ready to bundle up and deploy.
If you develop on Linux and deploy to Linux, don't you feel you will catch kernel and other OS-specific issues much faster? i.e. Before they become a problem in production?
Isn't it obvious?
With docker (or vagrant, or at least a VM etc) you can have the SAME environment as the deployment one. If you run OS X or Windows your direct local installs with differ in numerous ways to your deployment. And same if you run Linux but not the same distro or the same release.
And that's just the start.
Who said you'd be working in only one deployment/app at the time? If you need two different environments -- it could be even while working on version 2.0 of the same web app with new technologies--, e.g. one with Node 4 and one with Node 5, or a different postgres version, etc, you suddenly have to juggle all of these in your desktop OS.
Now you need to add custom ways to switch between them (e.g. can't have 2 postgres running on the same port at the same time), some will be incompatible to install together etc.
Without a vm/docker you also don't have snapshots (stored versions of the whole system installed, configured, and "frozen")...
I pass the repo, with the Dockerfile and docker-compose.yml over to another developer and they do the same thing. They don't spend hours getting node/postgres/redis/whatever set up then fight environment issues to match what my, or staging, or our production environment are.
> It was a prerequisite for any VM stuff being sold in the App StoreFor example I use a single docker installation in a VM to test several unrelated projects with all of them providing a web server on a port 80/443. I do not want to remap ports not to deviate from the production config. Instead I added several IP to the VM and exposed relevant containers on own IP addresses. Then for testing I use a custom /etc/hosts that overwrites production names with VM's IP addresses. This works very nicely.
But I do not see that something like this is possible with "Docker for Mac".
Guess what? I can do that on Windows 10 too and it works like a charm.
I'm actually holding up a wider scope for Docker deployment within our organization until we can use Docker for Mac.
The vagrant-based docker machine system needs some work to be a smooth experience, such as NFS workarounds for shared file system performance, file watching and so on.
This is hopefully avoidable in Docker for Mac!
Generally anyone who cares enough to ask directly, we'll automatically add to the top of the list.
EDIT: or, feel free to contact us privately with a few details on your configuration and use case: feedback+hn@docker.com
Thank you!
Thanks.
Thanks!
Thanks!
Thanks!
Thank you very much.
Thanks! My docker ID is: kevinmtrowbridge
Thanks!
My ID is the same as my name here, karunamon. Thanks!
Would love to try it out.
Docker ID: gtds
"cbushko" is the ID.
Specifically, I'm interested in exploring integration with JetBrains IDE's docker plugins
It seems like this is the future and I'd like to get an early peek so I can help my team move once it's release publicly.
Thanks!
Docker ID: rockymadden
Docker ID: travismcchesney
We use Docker to test our app against multiple relational databases.
My personal docker ID is chrisbuchholz.
Thanks.
I'm a student using Docker for many projects, including one for my university. Thanks!
Using for php and rails apps with mysql, postgres and redis.
ID: mentalpiracy
Trying to clean up our local machine dev setups
ID: pelotom
docker id: nivviv
Would be awesome to be able to try it out. Right now VMWare Fusion with a full linux and Docker inside. Had problems with the filesystem integration when using docker machine.
I have been checking my email for the last 30 days hoping to find an invite.
Thank you.
I've gone through and vouched them for you, so now they will show up, but new ones will have the same problem. Perhaps you can modify each reply slightly to avoid tripping the detector?
shykes, it might be useful to add a note that people need to sign up to the beta first!
Would be awesome to be on the Beta. We're currently migrating a lot of Atlassian related stuff to Docker.
Goal is to figure out local-dev workflows (ruby/rails + go/misc) and replace a vagrant VM setup.
Use Case: packaging up Rails/Elixir/Ember.js into a usable dev setup
Thank you
My Docker ID is 'killercup'.
Thank you all for investing your time in this thread, it is incredibly rewarding for the team to see their work appreciated and used by our fellow hackers. Constructive criticism with suggestions for improvement are the absolute best, thank you for those in particular.
Docker ID: szlend
Creating a local dev env of our stack and for running tests
Docker ID: jzvelc
Docker ID: zenitram
docker id: kevinparrott
Docker ID: lyndsysimon
Thanks!
Thanks!
Docker ID: joelcox
Docker-id: perhak
I'd love to test this out as a replacement for the Vagrant + VirtualBox set up we use currently to manage the development of PHPCI. :)
Thanks!
Would love to try this out on my mac. Eagerly waiting for the confirmation mail since I registered. Use case is to transform my development environment and get rid of all those local installs.
Thanks!
Thanks!
Really excited to see things built on top of Hypervisor.framework. I work with genomics researchers and I'm always encouraging Docker images for reproducible analyses. Starting workflows on OS X that can scale beyond laptops is really interesting! Thanks!
hopefully not too late - but thanks regardless!
Signed up on March 25th and continue to check my inbox every day. So keen to get my org. using Docker but the dev experience has been a blocker to date so very eager to get them on board!
Can't wait to see power of new Hypervisor. I student, played with Docker for a year, and this seems like a really big deal for me, so I can neatly organize my whole development environment and learn basics of containerization and deployment along the way!
Gonna be nice to get things started!
Super excited to get my team on the beta!
Thank you!
edit thanks for the correction netheril96!
I'm curious about the state of compatibility because I've drawn a line in the sand -- and refuse to upgrade from 10.9 (since many things seem to be getting only worse and less stable in MAC OS land :).
OS X Hypervisor Framework, which this uses via xhyve, is only available starting on OS X Yosemite v10.10.
https://developer.apple.com/library/mac/releasenotes/MacOSX/...
If it really worked (especially on Windows) Docker would post the binaries instead of treating this like Wonka Golden Tickets. Love Docker, am actually waiting to be approved so I can get to building something, but posts like this are a symptom of a larger problem.
That's why it's a beta.
The reason we are keeping the beta private is because we don't believe the quality is good enough yet to "open the floodgates". We are sending as many invites as the engineers are comfortable with - currently that's several thousands per day. As we hit more and more edge cases (performance, stability, support for unusual configurations...) we are expanding the pool as fast as we can.
Author of the post here.
I'll second your ask for patience.
If you've been following along, incredibly smart people have been experimenting with Docker and xhyve for a long time now.
I feel like I've been waiting for close to a year for a solution this good. Testing and trying lots of permutations on a Mac.
I was very excited by the beta announcement and was fine waiting for Docker to send invites at whatever pace makes sense for them.
I'll also still patient while the Docker team works out the countless new bugs they will find as people use this far and wide.
I'll also keep doing my part to help people be very successful developing and deploying with Docker.
Thanks for the hard work.
I'm deep in the weeds with Docker, LXC, containers, hypervisors all the time. Content about those layers very few people care about... These are tools that are supposed to mostly stay out of the way after all.
Interest in Docker packaging their app as a nice mac app, and people understanding how the install process works is not a problem, it's a sign that these tools are finally becoming digestable by all.
What's the technical error?
But note that beta 8 was released a few days after that review and already introduced some changes.
Also, for the Windows beta, it very much is still a beta.
To the users: if you can't configure a simple local web server for development, you should not be qualified to develop a web service. Period. (Hint: Apache and PHP is included in OS X, but a bit outdated to be fair - but you can install/upgrade to a OAMP stack using MacPorts in MINUTES)
And whenever a "newer, smarter company" disregards fundamental IT security practices and gets hacked, the sun shines a little brighter in my world.
Security is an afterthought (if a thought at all) in many hipster operations, and it's about time someone fucks up so badly that IT security is priority #1 from the beginning.