There's a major difference between the data-plane (the lanes where frames go from the signal source to the display) and the control-plane (the command and control channel that is bidirectional and used to discover a peer and negotiate settings).

The former is indeed very high speed, in the order of gigabits per second (up to 48gbps in later iterations), whereas the latter can seem ancient in comparison - 500bps to 115.2kbps - several orders of magnitude slower. The reason is that while the high data rate bus is handled by powerful, dedicated hardware (GPUs and ASICs/FPGAs), the control plane is usually done by micro-controllers (or otherwise lower-power devices) that are responsible for general management of the display. Sure, it would be nice if they had more powerful hardware and a faster bus, but those changes usually aren't what people are shopping for and so the standards committees aren't pushing makers to make these changes...