I have a nearly maxed out Mac Studio and my experience was the same, until I got my work to fork out a license for parallels and I just installed docker in an ubuntu VM and configured my DOCKER_HOST on the host to talk to the vm. Now it's crazy fast.