One of the comments I made to myself was that on the die shots, USB3 part is as big as a CPU core ! What would need so much silicon ? Isn't USB3 hardware more or less a serial bus? Is it logic or memory or analog interface parts here? I am curious.
There's a great deal of communication that happens outside the operating system, and a bunch of different multiplexing modes and things like that. There's at least 4 different speeds it's going to need to talk at (1.5mbps, 12mbps, 480mbps, and 5gbps) with a good deal of synchronization and other bits. There's a rather decent amount of logic that has to be implemented, and I wouldn't be shocked if you end up needing what basically amounts to another CPU for the controller itself to manage the entire connection and negotiation. Depending on the physical construction it might also need some decent power transistors if it's also handling the 2 amp current limit on usb3.0 internally (rather than signalling ones outside the chip).
The popular Cypress/Infineon FX3 USB3 controller has a 200 MHz ARM926EJ core on chip.
Nvidia at one point used one of their home grown Falcon cores (which looks like it could be around the same gate approximate gate count) to baby sit the USB3 phy as well. They're probably going to switch to RISC-V for that if they haven't already.
Doesn't the SoC include the actual USB-A connector?
It has to implement all the old versions of USB too