You also need space for wires, ..., etc, right? It's not just transistors.
However you can't always pack the transistors as dense as you would like because you can't fit the wiring for them in above at the same density.
Plus there are various 'design rules' that constrain how things get placed. These are needed to ensure manufacturing is successful and achieved good yield. An important set of rules are the 'antenna rules' that requires the insertion of antenna diodes (using silicon reducing transistor density) to prevent circuitry being destroyed during manufacturing: https://www.zerotoasiccourse.com/terminology/antenna-report/