> Nvidia and AMD seem to start with the mid and big dies first before filling out the small ones later.

For GPUs, yield is less of an issue... the chips are manufactured with the expectation of a number of cores of the many thousand small (in terms of silicon area) ones being defective - overprovisioning, basically. That allows them to simply bin the sliced chips according to how many functional core units the individual chip has.

In contrast, even the Threadripper AMD CPUs have only 64 large cores which means the impact of defects is vastly bigger, and overprovisioning is not feasible.