Another limit, now that I look, is the amount of power available to a PCIe card
CPUs can draw 200W or so which already seems extreme for a small rectangle of a few centimeters. But also consider that they run at low voltage, typically 1.1-1.2V or whatever. That means those tiny BGA/wires are carrying 100+ amps, similar to what a car battery delivers on starting. PCIe tracks are much bigger.