Having substantially more L1 and L2 cache per core but no L3 has to be a massive part of why the M1 performance is so good. I wonder if Intel/AMD have plans to increase the L1/L2 size on their next generations.
M1 has a massive reorder buffer, so it needs and can use more L1 cache. It's pretty much that simple.
Increasing L1 and L2 make intuitive sense.
Why this might help, ultimately, because the latency for getting something from L1 or L2 is a lot lower than the latency from L3 or main memory.
That said, this could hurt multithreaded performance. L1/2 are used for 1 core in the system. L3 is shared by all the cores and a package. So if you have a bunch of threads working on the same set of data, having no L3 would mean doing more main memory fetches.