The significant cost on Windows is due to having to spin up a Linux VM in the background, right? So the cost is probably more due to the VM than Docker itself?
1. is the VM 2. is handling filesystem mounts across operating systems
(2) can be debilitating in dev environments on Mac for example where you want to edit files outside the container but have the container have access to them. Docker have done a lot to improve this over the years but it is still very painful.