That's not how base clock works. Base clock is not min clock. Base clock is what it's "guaranteed" to hit under sustained load if TDP is respected. It's the TDP clock. A 4ghz base clock CPU will still be far, far below that when idle.
I went from an 6700HQ (2.6 Ghz base) to a 10710U (1.1 Ghz base) and the difference is definitely there, and it's jarring enough to the point where I kind of regret it. It feels like a huge step backwards, despite the latter CPU being four generations ahead.
This does not seem to be the case for my i7-8565U nor my older m3-something. The 8565U for example feels just as snappy as my desktop i7-6700, except it’ll throttle after about 30s.
Isn't this completely down to the cpu frequency governor in the OS?
My impression was that this "stepping" is customisable, at least in linux, I regularly step my CPU manually even.
I'm not sure what Apple has here, but maybe it's not what you expect.