Try to use less cores. RAM bandwidth is real limiting factor there, so there always some sweet spot between CPU cores and RAM bandwidth for individual system.
For example, I use only 6 cores from 10 on my M1 Pro laptop.
For example, I use only 6 cores from 10 on my M1 Pro laptop.
Use a maximum of the number of physical cores and then scale down.