Hacker News Reader

Comment by manacit | Hacker News Reader

I am not sure that this belongs on the front page of HN, but the problem seems to stem from the `virtio-blk` driver they're using not supporting TRIM, which causes space to never deallocate once the VM writes to a block.

Switching to `virtio-scsi` and sending regular TRIMs could fix this issue. It looks like they're also allowing people to configure the maximum size of the qcow2 image, which puts a hard upper bound on how much space the VM will take.

View on HN

(I'm a Docker employee working on this very issue. I guess my plan to take a break by reading HN failed!)

You're completely right -- it's a problem caused by lack of TRIM in the storage path. In the next beta of Docker for Windows (beta 31 due today hopefully) TRIM should be enabled. The Mac will take a little longer as we need to switch protocols and do more work on the host side -- unfortunately the default Apple filesystem doesn't support sparse files so we can't "cheat" by simply passing the TRIM down to the filesystem layer. We'll probably need some kind of explicit block-level compaction to shuffle blocks from the end of the file into holes that have been created by TRIM.

Keep in mind the Apple file system is going to change early 2017! (APFS)

It's going to be a long time before it's widely deployed.

I'm looking forward to trying APFS, particularly support for sparse files. But you're right, it'll take while before we can rely on it.

Does virtio-scsi and TRIMs work around it?

We'll probably use AHCI since hyperkit https://github.com/docker/hyperkit (the Mac hypervisor based on xhyve based on bhyve) lacks a virtio-scsi implementation at the moment. When the VM sends a TRIM ("these blocks aren't used any more, you can free them") we need to make use of this on the host. On Linux/BSD you could tell the host OS that the blocks aren't needed in the file using something like FALLOC_FL_PUNCH_HOLE and the kernel would take care of it e.g. by send TRIMs to the real storage device or shuffling filesystem metadata around. Unfortunately the Mac filesystems in common use don't support this so we'll need to manage this ourselves, e.g. by moving blocks from the end of the file into the middle and shrinking the file.

Maybe it doesn't but it makes the docker Mac port basically unusable for anyone that runs it on a laptop without much space free.

But wait there's more... the mounted volume performance makes it unusable for anyone using mounted volumes for stuff like say dev work. It's a complete joke.

https://forums.docker.com/t/file-access-in-mounted-volumes-e...

Reply on news.ycombinator.com