Edit: according to ChatGPT that is basically the reason. That one port is connected to the SoC’s building PHY that’s guaranteed to be on without needing any firmware. Other ports are routed through other controllers and whatnot and those may require firmware. Also the DFU port is guaranteed to not need PD negotiation to turn on.
DFU could opportunistically try to load firmware and start those devices but it’s risky since the firmware may be what’s bricked and might itself break DFU so for simplicity it’s in an absolutely barebones mode that the CPU supports and is wired for directly.