There's no reason a whole Thunderbolt bus is needed for every two displays. It's just Apple's decision to build their GPU that way.
And to not support industry standard NVIDIA GPU on ARM Macs, too. 1 GPU typically supports 5 output over as little bandwidth as PCIe x1.