> However, I'm still not sure why we need ultra-low-latency.

Simply for passive viewing probably not. For live streaming in which the viewer is also using actuators, e.g., gamepads, wheel, etc., for controlling the stream capture device, then yes.