If you want to remove a lot of unnecessary pain in your life, completely abandon the idea of mounting a host FS within a guest VM. Doubly so if they're different OS's/FS's and triply so if this a development VM.
My setup is now Remote VS Code to a Linux NUC that sits on my desk. All code lives on the NUC and all tests run on the NUC, but I can use my MacBook Pro as I move around the house, work outside, etc.
It's pretty seamless, I'm impressed.
I originally did this in a Linux VM, but got tired of the battery hit that running a VM 24/7 has.
It’s a bit of a pain to set up, and you lose some of the advantages of using Docker in the first place, but if you absolutely have to use it, it can get you back to full performance.