M1 uses TSMC high-density rather than high-performance. They get 40-60% better transistor density and less leakage (power consumption) at the expense of lower clockspeeds.

Also, a core is not necessarily just limited by power. There are often other considerations like pipeline length that affect final target clocks.

The fact is that at 3.2GHz, the M1 is very close to a 5800X in single-core performance. When that 5800X cranks up 8 cores, it dramatically slows down the clocks. Meanwhile the M1 should keep its max clockspeeds without any issue.

We know this because you can keep the 8 core M1 at max clocks for TEN MINUTES on passive cooling in the Macbook air (you can keep max clocks indefinitely if you apply a little thermal pad on the inside of the case).