(and in particular you can also push them off to N6 or some other super-cheap node and save your super-expensive N5 allocation for the GCD... at the cost of some power for infinity fabric to send a TB/s of data around between chiplets)
But so far nobody seems to have progressed on splitting the GCD itself... which has always been the actual goal here. Multiple chiplets computing together like one, transparently without software needing to be catered to like SLI.
AMD's general approach seems to be "big cache-coherent interconnect between GCD chiplets", which is generally unsurprising on multiple levels (it's how infinity fabric already works in their CPUs, and it's how SMP generally works in general today almost everywhere) but there still seems to be some barrier to making it work with graphics. There are of course a lot of gotchas - like temporal data or needing access to other parts of the frame that may be in another GPU - but cache-coherent interconnects generally solve this.
But yeah, NVIDIA isn't avoiding some silver bullet here, they've actually been on the forefront of this with NVSwitch since Pascal, that was the first serious "multi-GPU that acts like one" attempt. I don't know why they're not doing it, or at least the "split the IO dies off" thing.
edit: thinking about this a bit more, one reason they may not be doing the MCD/io die idea is because they're doing L2 instead of L3. L3 can be pushed off to a separate die... L2 is really part of the SM, it's a lot "closer". Again, they may have engineering reasons to favor L2 over L3, or it may be a patent thing.
I always thought the situation was frankly pretty silly, because I didn't want one big die anyway. I wanted a scaleable grid of chips like the 3D processor from the Terminator 2 movie.
It took over 20 years, but maybe we'll have a chance to get something like that with an open source RISC-V. Maybe an array of matchhead-size CPUs at 1 GHz costing $1 each that could be scaled to hundreds or thousands of cores with content-addressable memory for automatic caching and data locality. Then it could be treated as a single processor with access to a one large shared memory space and we could drop the (contrived) distinction between CPU/GPU/TPU/etc and get back to simple desktop computing rather than dealing with the friction of vertex buffers.
My guess is that Nvidia doesn't want any of that to happen, so works to maintain its dominance in single-die solutions.
The smaller you split the chiplets, the more data is moving around. And the more power you'll burn. It's not desirable to go super small, you want some reasonably-sized chiplet to minimize data movement.
Even if you keep the chiplets "medium-sized" and just use a lot of them... there is still some new asymptotic efficiency limit where data movement power starts to overwhelm your savings from clocking the chips lower/etc. And there's copper-copper bonding to try and fix that, but that makes thermal density even worse (and boy is Zen4 hot already... 95C under any load). Like everything else, it's just kicking the can down the road, it doesn't solve all the problems forever.