Would it ever be possible to route two stream via different paths to the client and let the client just accept the first packet from either and drop the other in order to add some redundancy to delivery?
A diagram of the setup on your page would be helpful...
Sometimes network paths die. This could be a dodgy router in a third party network that drops streams for 150ms at a time, or a bgp recalculation that knocks it out for maybe a minute or so.
In both cases you need to have multiple routes to keep your latency low.