When I built my most recent home GPU server, it seemed like 3U wasn't tall enough to fit a big GPU upright, given the PCIe power connectors on top of the card, so I went 4U.

Though I think you could get a double-wide GPU into a 2U with a PCIe riser cable/card.