If you're doing low latency audio or flight controls on an aircraft for example, you would absolutely need to abandon the fair scheduler for one that could guarantee the deadlines you need are met. Sacrificing "performance" that is throughput/efficiency/etc of your processor for timing performance.
Think of the difference between doing statistics on a billion lines of data and flight controls on a missile. VERY different needs for scheduling, why not have a selectable algorithm?
Not the right way to think about it; the connection latency is irrelevant. What is relevant is that you need to play audio in sync with the video, and that audio is coming to you approximately simultaneously with the video it's meant to be synced with.