With respect to HPD, one contributing factor per DP 1.1a § 5.1.4:

> There is no mandatory time constraint on the Source Device’s response to a plug / re-plug event, but a Source Device vendor may want to impose a voluntary constraint similar to that for HPD IRQ events (for example, 100 ms) to ensure a good user experience via prompt discovery and configuration of newly attached devices.

There's a lot more going on behind the scenes than mere "scanning for a valid data frame and flinging bits onto the glass", and much of it has nothing to do with approaching technical limits, e.g. the dynamic of multiple product vendors finding incentive to reach consensus in standards development despite naturally seeking to cut each other's throats in the open market.