Hacker News Reader

Comment by dchuk | Hacker News Reader

So is there some canonical guide to running a docker compose style app on Mac m1 machines that has good filesystem performance? It seems like there’s many ways to approach the topic now so it’s hard to tell which one is “winning”.

I’d love to containerize all of my local development efforts (scripts and rails apps) but the slow ass filesystem always ruined it in the past.

View on HN

We're using Multipass + Bravetools (https://github.com/bravetools/bravetools) / Docker to build system/application containers.

I believe multipass uses SSHFS (https://discourse.ubuntu.com/t/how-to-improve-mounts-perform...) to mount filesystems between the host and the VM. Performance has been excellent.

Because docker for Mac has always kind of sucked, I’ve moved on to making nix-based development environments and I’ve been very pleased so far.

Lately I’ve been trying https://devenv.sh/ and it works great!

I haven’t tried it for ruby, though I have used vanilla nix shell for ruby projects before and it worked quite well after I over-rode GEM_PATH and GEM_HOME to the correct values.

I’ve been using Nix for a few years and really like the direction of devenv. I much prefer the Nix model over containers everywhere.

I've been using Colima lately and it's been a great experience. The only thing that didn't quite work was pushing an image to AWS ECR, but pulling images, building images (with BuildKit), and running containers all worked fine, and faster than Docker.

I hadn't heard of Colima before seeing your comment, decided to give it a whirl and it's a breath of fresh air. Much faster and uses less resources. Thanks!

I've tried Colima too but it has always felt sluggish compared to local dev. I don't think any form of cross-host filesystem sharing could ever be fast enough.

isn’t Buildkit Docker?

https://github.com/moby/buildkit it looks like it is not, kind of?

Moby is Docker.

Whatever it is, Colima seems to have faster image builds and faster container startup, which makes my life better.

While it looks really nice, "Without containers" at this point in time with the supplychain attacks doesn't seem to be a good thing.

Can devenv environment put itself in a chroot / sandbox where it cannot access my files, and is firewalled by default?

help me understand

nix file helps you build OCI image and then you run it not locally/somewhere else?

can that be compared apples to apples to docker where you can build with Dockerfile syntax, and then run locally the image?

Nix builds native software for your platform reproducibly. There is no OCI image, no Docker, just a way to reliably install software, which is a great foundation for building per-project development environments.

(Aside: you can build oci images using nix, but that’s probably what you would use for building deployable artifacts for production, not for development).

I would not say it is comparable with a Dockerfile - a Dockerfile is closer to a shell script, with caching between each step. Credit to Dockerfile syntax for being very easy to grok, but it is inherently not reproducible.

The perceived reliability of docker for development probably comes from the fact that you can share a prebuilt image, but rebuilding might not always work.

I've recently switched to using nix-shell and overmind with a Procfile to officially retired docker-compose from all of my projects. Such a breath of fresh air.

Like Mitchell Hashimoto (Hashicorp), I no longer do any development on macOS directly; I have a Linux VM where all my projects live and have their environments managed with a shell.nix file. Highly highly recommend it.

Would you elaborate on the setup? I recall seeing a comment or article where Mitchell mentioned this setup previously. I’ve been using Make + Docker for development on macOS for a long time now and perhaps it’s time for a change. I’ve looked at nix once or twice over the years but it never really clicked.

Sure, this is a super abbreviated version of my current setup for developing https://notado.app

shell.nix - this brings the packages I need to develop into the $PATH when I run `nix-shell` in the repo

    {pkgs ? import <nixpkgs> {}}:
    with pkgs;
      mkShell {
        buildInputs = [
          meilisearch
          openssl
          overmind
          pkg-config
          postgresql
          rustup
          tmux
        ];

        shellHook = ''
          export OVERMIND_PROCFILE=Procfile.dev
          export PGDATA="data.pg"
          export MEILI_DB_PATH="data.ms"
        '';
      }

Procfile.dev - this is used to start the background services I need, the format is "name: command"

    postgres: postgres -K /tmp
    meilisearch: meilisearch

Then I can run `nix-shell` to load all the dependencies and `overmind start` to bring up Postgres + Meilisearch with ignored data dirs relative to the project folder. If any of these packages need to be pinned to a specific version you can do that with this tool[1].

That's pretty much it - with all the background services running you can load your monolith or microservices with cargo/npm/etc. It's all very nice and clean!

[1]: https://lazamar.co.uk/nix-versions/ - I actually just used this in a new shell.nix for my personal website to pin a version of Hugo from 2019 (when I last touched it) so that it would build again

this is great, and I never heard of Overmind until now (looks cool!) but you could also flake this config out and get a couple more perks, like possibly not having to use the nix-versions thing

What do you use to run the VM? Do you try to minimize overhead at all?

My general setup mirrors this[1] pretty closely, though of course on Windows I can just use WSL2.

[1]: https://m.youtube.com/watch?v=ubDMLoWz76U

If you’re using nix, why are you using a VM? Nix on macOS works pretty well these days.

VMs are better for my particular use case as I am regularly shifting between my Macbook and my Windows desktop with WSL2.

Have you tried enabling Virtiofs in Docker Desktop? On my M1 Mac that sorted the performance for me for large PHP apps, and I haven't experienced the permissions problems that some people have (there is a test build that appears to fix those out now).

I have an M1 Pro MacBook and I turned on the 2 experimental settings and that sorted my performance issues.

I think the new virtualization framework just came out of beta though so now it’s just VirtuoFS that needs to be enabled.

> slow ass filesystem always ruined it in the past.

Make sure that you're running a recent version of Lima, and use a 9p mount.

Edit: by editing one of these yamls: https://github.com/lima-vm/lima/tree/master/examples

It's less than perfect and being worked on afaik... Main thing to avoid are any volume mounts to the host environment. Declare/use volume containers in your docker-compose.yaml, will help a lot, but still more sluggish than host/native.

If you use VS Code the remoting extensions pack includes SSH, for my personal laptop, I'm usually using code to my linux desktop, and editing remotely... it's pretty slick. Code will run a "code server" in the remote environment, and is effectively just displaying the text view to you, the actual work is happening in the remote envornment. You can do similar with containers, or in windows+wsl.

I'm thinking about giving this a try but using an iPad connected to a monitor setup for this. I rarely code on the go for personal use, so this could work pretty well...

There are codium forks that will host a web server environment for you... easy enough to VPN home, or another trusted environment, and use a code(ium) server on your host.

Canonical has a product of their own: https://multipass.run/

They all use the underlying OS hypervisor so I’m not sure you can get perceivable performance out of one solution over the other.

Sometime it can be buggy on macOS and quite slow to boot with Ubuntu VM, wish there is a cloud VM image.

> wish there is a cloud VM image

What do you mean? Multipass only runs Ubuntu server cloud images.

Reply on news.ycombinator.com