As far as I remember, doesn't Docker for Mac create a VM and then run the docker containers in the VM? That'd explain the terrible filesystem performance.
It does, it's a lighter VM than VirtualBox which is what docker machine used, but still a VM nonetheless.
docker-toolbox you are talking about. The new version (Docker for Mac) with this problem doesn't use a VM anymore, but native virtualization on OS X. The old version doesn't have this problem.
Native virtualization is a VM...
xhyve spins Alpine Linux VM inside which it runs Docker Engine. So basically it don't deviate too much from running Docker prev. to docker-toolbox for macOS.