On some level yes, on some level no. The physical interface for each point-to-point link is serial, but one device can have several of those links and use them (send/receive packets on them) in parallel.
Switches are far more complex. The protocol specifies addressing, bandwidth reservations, flow control, priorities and message ordering requirements, which the switches have to handle.
Not a switch thing per se, but also relevant for "single transistors won't do": On the physical layer, each port/link is also relatively complicated to initialize and operate. The transmission encoding, due to the relatively high speeds involved, requires a learning phase where both ends measure the link characteristics and adjust their (relatively complex) signal processing accordingly. The physical layer also handles different link speeds for down-/upward compatibility.
This may explains how they achieve the graphics blitting in the Vision Pro.