I'd argue that on a lossy network you will never have decent video (with decent latency) so for interactive sessions you don't really have to care about lossy network, it isn't worth the hassle anyway.
Ever since h264 there is a feature called intra-refresh, which allows you spread key frames over time, reducing the burstiness of the transmission (and therefore improving latency in most scenarios)