If the connector is high bandwidth enough to send hundreds of thousands of pixels 60 times a second, how can it possibly be so slow to send a few bytes of handshake or EDID data?
Handshakes should be happening nearly instantly. You can complete opening a TCP connection to the other side of the world in a fraction of the time it takes to switch input on the average monitor.