Speed boost achievement unlocked on Docker Desktop 4.6 for Mac
docker.com
docker.com
Nice to see the fixes make it into a release!
PS: of course you can support Docker also: https://www.docker.com/pricing
Hehe, not if you run Docker in a Linux VM, edit your code using the SSH VSCode plugin, and port forward any services from the VM to your Mac.
After doing this for over a year, I don't know why anyone still uses Docker for Mac, other than corporate how-to guides that still start with "1. Download Docker for Mac", or feeling unfamiliar with Linux.
Docker was originally created as a chroot'ed process container with abstracted networking, nothing more -- and certainly not a full VM. Why not run it as it was intended to be run?
Edit: Am I incorrect? Or is something else wrong?
The thing I don't like about your method is having all my code checked out on the VM. As soon as I want to start using anything other than VSCode to manage it, I'm now hopping through layers. I'm also restricted to the VM's filesystem/size limits, and individual changes to files are not backed up by Time Machine, only the entire VM disk image.
And only because you mentioned corporate how-to guides, Docker Desktop requires a paid license for commercial use. I think your method is a perfectly valid way to work around needing a license, but the license comes with commercial support and some other features some companies may find useful.
Isn't that the usual case: you have your code on your local filesystem and then run it using an interpreter in the Docker VM?
Well sure, but almost nobody is running production workloads on macOS. Regarding development workflows. Won't rebuilding the image every time you make changes be slow? I can get ~1 second reload times for node.js services outside of Docker. And I believe inside Docker too on linux.
However, they develop on them - and it makes sense to make those environments (and the processes) similar to your actual production setup. So if your applications code gets compiled, and deployed as part of the Docker image - then ideally when you spin up a dev environment, it should follow the same process.
The whole "magic" is that you work on the files on your host machine! I think VSCode is doing god's work (along with other tools) to make remote editing a nicer experience, but well... I use Docker on Linux and just edit files locally. It's nice!
IdentitiesOnly yes
AddKeysToAgent yes
ForwardAgent yesVagrant takes a lot more disk space and RAM. VirtualBox isn't as efficient as the macOS hypervisor. And every Vagrant project is a separate VM, instead sharing the same VM for multiple projects.
We find it significantly faster to do macOS + Linux VM + Docker than macOS + Docker for Mac. We're using the Remote Containers plugin in VSCode.
I just tried Docker for Mac 4.6.0 and it is still slower than the above setup - though I think this might be down to bind vs. volume mounts.
Before anyone mentions sshfs, macFUSE+sshf is literally the only thing I’ve seen that can fuck up the entire disk subsystem’s ability to mount and unmount anything until you reboot.
[1] https://github.com/kata-containers/documentation/blob/master...
[2] https://www.codeluge.com/post/setting-up-docker-on-macos-m1-...
[3] https://www.lifeintech.com/2021/11/03/docker-performance-on-...
In more interesting setups, the class files aren't in the image but rather mapped in - much the same way one would with dynamic and then a hot reload - https://docs.spring.io/spring-boot/docs/1.3.8.RELEASE/refere...
> Spring Loaded goes a little further in that it can reload class definitions with changes in the method signatures. With some customization it can force an ApplicationContext to refresh itself (but there is no general mechanism to ensure that would be safe for a running application anyway, so it would only ever be a development time trick probably).
And this way, the container can remain the same with the class files being changed underneath it.
I'm aware of some document describing that it's not really a resource leak, and instructs us to look at real mem instead of memory consumption, but sadly MacOS is not conviced, and will happliy use memory and swap space until eventually it crashes with out of memory, so while a memory leak might not be the cause, _something_ is claiming to use a lot of memory and not releasing it again.
Instead 9p experimental support got there a few days ago: https://github.com/lima-vm/lima/issues/20#issuecomment-10660...
- On MacOS host: around 2s - Inside Docker container with VirtioFS enabled: around 20s - Inside Docker container without VirtioFS enabled: around 25s
When these happened, I can't even do CTRL D in the terminal. The STOP button in Docker doesn't work. Had to restart.
Anybody else?
Otherwise it does seem quite q bit faster, like `vite` starts up almost instantly instead of taking a second or two (having been used days prior, so it has its cache).
But for desktop development docker is a godsend to have a reproducible environment.
In contrast, I largely stopped using docker os x because of the broken fs virtualization here made it impossible to do productive JS dev. Subsecond editing vs 10s-3min is a huge step back. Unclear if the the new patches get it to the native-level feeling of wsl2 (1-3% overhead) or still feeling broken in practice. And of course, apple's vertical control means no nvidia GPUs / CUDA, even eGPUs, so no way for docker os x to support that either.
One other issue this also solves is that volume mounts from windows don't inotify correctly, so hot reloading doesn't work properly. But with files on the WSL side that works fine.
Lets hope it will be possible to communicate more efficiently in the future.
I was very interested in this Bazel-based way of building containers but its README page says "it is on minimal life support," which does not inspire confidence. How's your experience using it?
Jib has always been absolutely excellent for Kotlin and Java, really can't fault it in any way. I specifically use it with Gradle though, unsure about the quality of the Maven plugin but I would assume great also.
> Docker Desktop 4.6.0 gives macOS users the option of enabling a new experimental file sharing technology called VirtioFS. During testing VirtioFS has been shown to drastically reduce the time taken to sync changes between the host and VM, leading to substantial performance improvements. For more information, see VirtioFS.
On Linux itself, Docker simply uses the same core Linux technologies to create the isolated runtime "containers" using "control groups" and "namespaces." Docker on Linux is basically native speed because it doesn't have to jump through hoops like syncing the VM filesystem with the host Mac's filesystem, which is a major point of slowness and a long standing pain point with Docker for Mac.
Do you know in simple terms why macOS doesn't have such "native containers"? After all isn't it more linux-like than Windows?
I saw this website that suggests Apple themselves are not providing such a feature... do they see it as a security issue?
A "control group" (or "cgroup") is a feature of the Linux kernel that lets you create an environment with a fixed allocation of memory, CPU, and other resources. Aka a controlled group of resources. Whatever runs in here only gets the resources defined by whoever created the cgroup.
The other feature is "namespaces", aka a "process namespace." It's another feature of the Linux kernel that lets you create an isolated namespace for processes to run in. If you're a process running inside this namespace, to you, it looks like you're the only process on the system. You can run multiple processes in a namespace (Docker containers can run more than one process!)
So on Linux, to create a container, you basically use native Linux features to build this isolated environment, and run some process(es) in it. This is also why there are different ways to run containers (BSD jails), really what we're talking about is building an isolated environment.
As far as I know, the Mac kernel doesn't have these same features to create isolated environments, which is likely why Docker for Mac went with a full Linux VM, which includes the kernel. This is approaching the limit of my knowledge. I don't know what the Mac kernel is missing or what gaps or proposals there are to create a container-like environment. I also haven't worked with BSD jails or other technologies so I don't know how portable containers are between systems, if at all.
BSD has jails which are similar but different.
So the answer is MacOS doesn’t offer containers like you know them because it’s not linux hence Docker for Mac spins up a VM.
The reason Docker Desktop and similar solutions need to come up with these different complicated solutions is that they want to share files between containers running inside a Linux VM and the host system. So they need some kind of way of syncing the files.
Only Little Snitch stopped it.
Note also that unless you are using Tor or something like Private Relay, your "anonymous" usage reporting isn't anonymous at all. Client IP is location data.
This is a straw man argument, just in case anyone cares. The "reminder" is that Docker isn't free and not "even" Open Source. I think Docker has produced Open Source code, however. As most of us get that Docker is nowadays NOT Open Source, the argument falls to "reminding us" that it "uploads a ton of sensitive info" "without consent". Whatever.
Docker has personal info on everyone who uses Docker and uses their online services. Phone numbers. Email addresses. Passwords for various Docker websites. People love this because they are Docker customers or long time users. They don't care because most of these customers/users are DOING BUSINESS. Sure, Docker runs third party analytics on their site and pixel trackers in their emails. Big deal. They are in business to make money. They aren't used by everyone (read the public), and most people that run it (assuming it didn't uninstall itself), run it on their work computer. What info is on these machines that Docker would risk their shareholder value to "steal" or "divulge"? Not much, is the answer.
To be clear, that last part of the straw man isn't a reminder, it's a claim. Further evidence of straw man shenanigans is present here in replying to "source?" by saying "the binary". Of course data would come out of a binary, but which binary are you referring to? The Docker binary, which runs containers and crashed, or the binary that handles the crash of the other binary? Or maybe it's the installer binary that crashed? Mine did. Twice.
All that said, all you people who just shout "privacy, privacy, grumble, grumble, large company, grumble" are a pain in the ass. If you care about privacy, limit what software you use, put stuff that you worry about on another machine and keep your damn data off it. Privacy advocacy isn't worrying about your privacy, it's understanding how the public, at large, looses access to understanding about where they should share their data. Just telling people to worry about privacy isn't helping, and probably does more harm than good.
Argument for or against what, exactly? It's a statement of fact.
Building software is also the thing that pays a lot of our bills. We expect our customers to pay us for our engineering labor. What makes Docker any different?
I've also found that a lot of large enterprises have had big difficulties on hammering out licensing terms with Docker. When it's a product you actively use, that means the devs will either wait for shit to hit the fan or start looking at alternatives, and find that there are still free competitors (Rancher Desktop, Podman, Minikube depending on your use-case).
IMO, it would have been a whole lot better if the Desktop product had been paid to begin with, rather than being suddenly switched as a last minute monetization strategy.
The first objective of a pre-revenue company is to grow as fast as possible and then either get acquired or start generating revenue, which almost always involves marking up their product. And because of the nature of software, it's easy to build POC's and prototypes that half-work today, with the promise of better functionality and integration in the future. But that tomorrow always comes with a cost.
Somehow, folks in this industry seem to be convinced that it's OK for them to turn a profit on their value-add software, but not other people. What are software companies supposed to do? Overcharge today for software that doesn't necessarily work right now? What's the model that companies like Docker should use instead?
You're mistaken here. Nobody has issues paying for software, however people have issues with being baited into a free product to then suddenly be charged after companies get locked in, which is a shitty business practice that needs to be discouraged.
Had Docker been transparent from the start that at some point in the future it will cost money, then that would have been fair since this would have been factored into business decisions.
You can't just hand out free candy and after people swallow it, you tell them they now have to pay up or throw it up, since you can't expect such good candy to be free. That's basically a scam.
Again, it's not about the money, it's about transparency.
Docker never said the product would be free forever. You'd have an argument if they initially committed to keeping Docker Desktop free forever and reneged, but they didn't. It's kind of silly to expect a company to communicate in such vague terms. What company comes out with a statement saying "we're not sure when or what the details are, but someday, we're gonna start charging for Docker Desktop". Also, how does this work when a company decides to pivot? Maybe they initially intended never to charge for DD but their initial business strategy didn't work and they had to pivot. Is it wrong for a business to modify it's revenue streams?
If you choose to incorporate a a piece of technology developed by a private company into your commercial project that is free today, you should expect to pay for it at some point in the future.
I can see why people aren't happy with how things have played out. Docker does need to earn its crust and pay its people, and this is strictly for the desktop UI, team space, and nascent collaboration feature with dev environments. It's a value-add over barebones docker but, and this is what is probably frustrating, it targets Windows and Mac users exclusively because it's not as easy to run docker without their software.
Let's not also forget that they made optional upgrades a paid-for feature, while at the same time releasing software with showstopping bugs and regressions. Their response was literally to pay them money so you could deny an automatic upgrade. So my sympathy for Docker finding itself in this position is fairly limited, knowing that.
The solution is to add additional features and charge for them; they should have charged more for e.g. automatic kubernetes configurations and other more "businessey" features, but now it's too late and there's no value-adds that they haven't already added.
In the end, it's not a complaint about Docker's "value-add" software; they're complaining about an increase in the price without an increase in the value.
I'm not sure what Docker could do from this point on. Maybe I'm missing something, but they seem pretty doomed overall.
You can provide something for free, not make any promises that it will be free with upgrades in perpetuity, and one day decide to stop. It’s as simple as that.
Says who? There's plenty of free as in didn't transact dollars for software that I have extremely high expectations for. Visual Studio Code, Gmail, Github, etc.
When they one day decide to stop I can also choose to be disappointed, not buy and no longer use their software. It's as simple as that.
If they made it explicit that it is free forever, then I would agree with your stance that Docker in the wrong. But so far I couldn't find a shred of information indicating that Docker commercial/enterprise account will be free forever. So that didn't put Docker in a wrong.
And be realistic of this world, you cannot expect software companies to survive on free account alone forever. Who is paying the servers to keep it up? Docker. Who is providing the support? Docker. Who paying the developers to keep Docker products updated and introduce new features? Docker. How could Docker survive without the cash flow with the free account forever. I mean Google, Microsoft, Chococately, Homebrew/Brew, etc have commercial licenses to cover the cost of their products to keep it free for personal-use users. If you don't see this coming, well I don't what to tell you but that is the cost of doing business.
Well sorry but free without any conditions attached, becomes free as in beer. If I give you a shiny toy for free no strings attached, I can't come back to your house later and ask for money for it because I didn't specify how long it would be free. If I didn't specify the conditions of the 'free', then I goofed up.
If they didn't put any conditions on the duration of the 'free' from the start and is causing them to loose money, then either they were financially stupid from the start or, more likely, they wanted to pull the classic SV success scam where you bait customers into a free* product for years, burning endless cash to capture the market, and when customers become locked in thanks to the 'free', and you are the de-facto standard with no competition left, start rent seeking.
And people complaining about the "bait-and-switch" that occurred few years ago while Docker has been charging for non-free licenses way before that. Huh?
https://web.archive.org/web/20200101044220/https://www.docke...
Would you rather they go under as a company, due to how badly their attempts at monetizing containers have failed in the past and then have Docker Desktop not be maintained at all and have Docker as a whole take a noticeable hit?
> ...and a non negligible cost.
I'd say that perhaps using either Docker with the CLI, IDE integrations/plugins or using Rancher Desktop (https://rancherdesktop.io/) are more cost effective alternatives in that case.
Now I have to use your software because you used investor money to make that software industry standard (standard, which is in fact pretty low). Yet you still made no money.
Who knows, maybe if Docker didn’t exist the vacuum in the solution space would be filled with something better or worse?
The initial setup is slightly easier, but then you're exposed to random modal dialog popups for all time moving forward.
If there was some value add, then maybe it would be worth paying for. As it is, it looks like they spun off the part of the company that had any hope of turning a profit (via support contracts), and now they're flailing around in search of a business model.
It's a shame. Dockerfiles are nice. I hope they come up with a way of getting people to pay for some (actual) value add.
I know there are better alternatives but it seems like there are a handful of alternatives right now and no clear preferred alternative.
Is that how it works for commercial license? Commercial license come with extras than what personal-use/free licenses have. You are whining that Docker is charging for commercial license that the company you works for is a commercial entity. Docker provides an platform and they are not free for commercial to use. Personal use is a drop in the bucket for them because personal users are not overwhelming their servers or pulling huge data daily. Commercial entities is a deluge in the bucket for Docker because entities use their servers more often than personal user do. Commercial uses consumes more data than personal use and you want Docker to accept that and eat the costs for commercial entities? If Docker does this, then Docker will not last a year without commercial pricing.
You, sir, have a strange ideology of this.
Another major part was SSO being merely on the roadmap at the point we would have had to begin the purchasing process with our licensing service. “Well, just slap down a credit card!” That is not how large enterprises work, and Docker appears to have fallen down on their market research when they didn’t set up a way to deal with purchase orders.
Third parties even produced patches to do this but upstream rejected them.
They wanted a monopoly on burning server and network time on CI jobs. Now they have it, and I'm not sympathetic if it wastes their money. I'm more concerned that it wastes my time.
I know that I would have to run Linux in a VM. My question was whether Podman provided an installer that set up a Linux VM in the same way that Docker Desktop does.
It's completely awful, seeing Mac and Windows spend their their trying to VM Linux, poorly. But that's what I have to work with.
The community edition of Docker is much closer to most cloud container setups anyway, so using Docker Desktop is really counter productive.
There are better alternatives for your workstation, especially if your production environment is Linux based. Most, if not all, of the major development environments run well in Linux. Obviously not XCode, but for server side stuff it doesn't matter.
1. That I don't use Linux when I can. I use Fedora on all my personal equipment, including laptops. When I have a choice I do use Linux because it is simply a better development environment. The question was because this is an article about Docker Desktop on MacOS and the poster suggested using Podman.
2. That running `brew install docker` was actually that hard.
Don't get me wrong, I absolutely dislike running MacOS on my work laptop, but I don't have a choice and I have not seen a job where I would have a choice. Docker works "seamlessly", except that being a VM it does use 3Gb for even the most basic containers.
They have never let me pick Linux.
[Edit] Actually, one did. That was also the first place that I used Linux containers, long before Docker Desktop was a thing, and before the hype train really got rolling. Back when Docker really made sense for process isolation, rather than using it as a lazy Linux VM for building on your laptop.
I would disagree that WSL2 is a clear choice. That would mean using Windows, which is a trainwreck of a UI, and gets less consistent with each release. Has Windows 11 fixed multi-desktops yet?
I personally find using Linux, with Gnome, so much more pleasant than either Windows or Mac. Much more consistent and without the arbitrary restrictions.
Keep in mind that docker for desktop needs a license for companies having more than 250 employees or 10 million in revenues, not from kids experimenting, not from hobbists, not students, companies doing 10 millions of dollars, if you are a company that makes 10 million of revenues a year and have uninstalled a software that was valuable for you to run away from paying 5 dollars, then write the name of the company so we can also avoid it in the future
If it was 5 dollars one-time fee for a greater level of access to the docker hub, many companies would have gone for it. At 1,000 employees you'd have to get $60,000 authorized yearly, which is going to make many companies look for the cheaper/free option.
Upon waking up some more, I found this: https://www.docker.com/pricing
"Teams" option is $7/user/month. "Enterprise" is $21/user/month.
In a recent gossip session with some buddies, one told a story of a company that had hit the Docker rate limit during a production deploy but still refused to pay the subscription fee!
Plus so much open source software that should be funded by companies whose business literally would die without it. How many companies fund Homebrew, or whatever apps are used in their build toolchain? How many pay for desktop or server Linux?
What's your solution for that? A bunch of full-fat VMs configured with Ansible or something? Been there, done that, less convenient, more resource-intensive, and far noisier/cruftier than Docker (though the script-configured-VM approach is still far better than running that stuff directly on my machine[s], admittedly)
If your production environments don't require reproducibility that is fine. If you want to figure out what might be causing issues in production by using a setup that has the same dependencies but a different underlying base, fine. But that just won't do for lots and lots of products.
Containers are better.