Ask HN: Can there be a true real-time collaboration over any distance on earth?

The first-person shooter crowd has probably run into these limits so that's a good market to go see what works and what doesn't. Your music example may be a little misleading. Although we can detect small delays (10 or 20ms) our reaction time for input is an order of magnitude greater (100ms or more), so we may be able to tell a note is off by 10ms but we only do it 100ms after the fact. Depending on what you're doing that distinction can be important. For human communication for example a 500ms roundtrip time would still allow a conversation easily.

Your math is also slightly off because the speed of light in fiber optic is only 200000km/s, so the minimum latency between two opposite points on the globe using fiber is 100ms.

PS: The circumference of the earth is also 40 thousand km not 40km, so I suppose the correct notation is 40Mm, although that's unusual.

> The first-person shooter crowd has probably run into these limits

Too true. Early on, when people were still connecting over modems, the ping times were pretty much guaranteed to be well north of 100ms, even to servers that were relatively local. As universities started getting equipped with expensive low-latency/high-bandwidth internet connections players started getting <100ms pings and as a result these "low ping bastards" (lpb) ended up dominating servers.

Very quickly hpb and lpb servers developed to serve and enforce a bifurcation of the two groups of technology users. Scripts usually ran to kick/ban lpb players from hpb server and to warn hbp players they were on an lpb server.

Importantly, if you get get <100ms ping times, the predictive engines in the games (designed to help "help ping bastards" or hpb players) solved most of the rest. A good player with a <50ms ping was virtually unstoppable.

You could also tell when certain players were likely cheating, Team Fortress snipers with >200ms pings but incredibly high kill:shot ratios were usually the form it took on the games I played. From their perspective, everybody was jumping around in huge gaps and prediction by the player, even once you got used to the lag, would have been virtually impossible.

I used to work at an ISP connected to a tier-1 provider connected to MAE-EAST and routinely had 10-20ms ping times and enjoyed high placement on leaderboards for quite a while.

More recently, with my in-laws now on the other side of the planet, and having spent some time conversing on Satellite Communication links, 500ms for voice communications is doable, but definitely disruptive.

On a sidenote, I checked pingtimes from my FiOS connection to Japan (yahoo) and to a local server and inside my home network.

Home Network: <1ms

Local site (10 minutes away): <25ms 12 hops

Japan: <200ms 16 hops

Even if the reaction time is higher than what we detect as a latency, it's still a bad news, but delivered ... later. :)

I left 300000 km/s, because I thought that maybe this is the absolute theoretical limit (there might be some breakthrough in reflective materials, who knows?).

Sorry for the typo of the circumference of the earth, I think I've fixed it before your comment. https://news.ycombinator.com/item?id=7862387

>Even if the reaction time is higher than what we detect as a latency, it's still a bad news, but delivered ... later. :)

It would be hard to do a distributed band. It's an interesting case because it's not a question-response situation where the roundtrip is crucial, because for the most part each musician is playing independently and only listening to the rest to stay in sync. I wonder if you could do this:

1) Have every member stream to every other member directly with an accurate absolute timecode (good NTP-adjusted or even GPS clocks)

2) Have every member mix in all the streams using the absolute time code and buffering as much as needed for the slowest stream (so you'd have 100ms of buffer in the worst case).

This would give you a synced up band where every member listens to the same mixed stream and yet everyone is listening to themselves on a slight delay. I wonder how well our brains could cope with that introduced delay and if it would then screw up live performance, or if we can adapt to that sort of thing quickly.

Maybe it would be possible if you can circumvent the human sensory organs and directly feed the signal to higher brain areas. The delays in sensory modalities are anyway very diverse and not at all synchronously processed in real-time. Even within the same modality the signal is disintegrated in the brain. For example an image from the retina is processed with completely different delays in the ventral and dorsal stream. The ventral stream has latencies of several 100 ms [1]. It is still an open question in neuroscience how these different latencies are combined in higher areas in a way that they are perceived as synchronous.

So even if you are in a conversation with people directly in front of you, you do not actually have a real-time system. It takes roughly 400 ms until the image of a face of a person standing in front of you is processed in the ventral stream and is actually detected. On the other hand there is the auditory modality which is much more sensitive to latencies. But in the end the brain gives you the illusion that anything you perceive is actually real-time, although it is not. So if your brain decides to give a motor signal to actually press a piano key, then only many 100 ms later the actual sound will arrive. The perceptible latency of hearing a sound is actually only because the brain itself takes into account the delays in itself.

So in the end it comes down to trick the brain into believing something is real-time. This is for example done in multiplayer ego-shooters where the motions of other players are actually interpolated into the future and later on corrected if the other player actually changed their last motion command.

[1] Barragan-Jason G, Besson G, Ceccaldi M and Barbeau EJ (2013) Fast and famous: looking for the fastest speed at which a face can be recognized. Front. Psychol. 4:100. doi: 10.3389/fpsyg.2013.00100

As a data point, an organist may have to deal with latencies between few hundreds to seconds. So it's not that we can't deal with latency at all.

Ninjam[1] arranges realtime music collaboration by exploiting musical structure and its construction in intervals. Human collaboration which can be similarly structured could still be arranged to work with.

Counter examples: games appear to work over latencies because a lot of them are about restricting other's agency and making an experience worse (i.e. not true collaboration) 2d fighting game can be seen for instance as two game designers attempts to make a game worse for the others. It would be interesting to study why games like left4dead (which requires team play and collaboration) still work.

[1] http://www.cockos.com/ninjam/

Even when jamming in my personal studio anything above 5ms latency on my electric drum set can be noticed. I have to lower the IO buffer size to about 64 samples to get above 5ms latency.

I would love to be able to jam with friends over long distances. For this to actually be enjoyable without frustration the latency would have to be no more than 5-10ms.

I wonder how much latency a local telephone call would use? Then maybe it is possible to set something up where you jam over the telephone line and each musician can hear each other just fine.

I wonder if this would change if you were never allowed to play an instrument with less than 100ms lag (ie: from the time you started practising)?

[I realize that such instruments as table tops and cutlery are readily available, but I wonder if restricting regular practice to an electronic instrument with artificial lag would counter this. Possibly the music would only be enjoyable to others who practised similarly..]

The sound quality of land line voice connections is terrible for anything but speech. IIRC it's something like 8 bit 3 KHz

One way to do this is to jam based on a loop, so everyone can be off by one bar. I think there are some services that do this.

Specific pointers? I'm curious how this would work in practice.

An interesting point to consider is that `(speed of sound) * 67ms ~ 25yds`. A musician could elaborate on the challenges involved in playing in a large symphony orchestra, and yet it's accomplished.

That's one of the reasons you need a conductor. Orchestra players learn pretty early that if you listen for your cue you'll come in late.

What a wonderful comparison. And it gives hope to the fact that software at either end could play the role of this conductor — it can negotiate a "true" sync time and pre-empt the delay, just like a conductor.

Indeed. Still, it might be enabling to someone who's already used to the delay, but in a 4-10 person band, there isn't such an issue.

I have to check something, though - how do things change if all musicians tracks go through a mixer and everyone listens to his own mix in headphones. On stage, the delay from the main speakers is huge for the perfomers. That's why they have either in-ear headphone or speaker monitor. Can't this work for symphonic orchestras? And does it solve their latency problem?

Symphonies are pretty traditional in their methods, and the conductor is already an adequate solution to the problem (and has many other roles).

I never really thought of it this way. Thanks for the great examples, guys!

What exactly is your application (and your budget)?

For videoconferencing, musical-instrument-playing (as in your example), or multiplayer video games with less latency than the speed of light appears to allow, you could take advantage of Libet's work (see http://en.wikipedia.org/wiki/Neuroscience_of_free_will) showing that with neural interfaces you can reliably predict people's physical actions hundreds of milliseconds before they actually do it … which would give you hundreds of milliseconds to send off the signal and get a response which you play back to them "instantly" when they perform the action.

I haven't started anything, but I always wanted to be able to have a band not restricted to the local pool of musicians and this is what made me think about it. So I started looking for a solution, I found about the Internet2 project and when I saw that they should be pretty close to the theoretical limit, I decided to check what that limit really is. That's when I realized that I probably couldn't do this for any earth-distance without any prediction of data.

In fighting games people are very sensitive to latency when you get to a high enough level. There is an interesting networking library called GGPO http://en.wikipedia.org/wiki/GGPO that cancels lag out using a clever algorithm. I imagine you could apply a similar technique to other problems.

The wiki article isn't very clear, but it sounds like GGPO hides part of the network latency by disguising it as the natural delay between deciding to act and the action taking effect, therefore delaying actions in the game in a believable way. Is that right?

There's an interesting demo of the effects of lag on something like VR. I think it's intended as a statement against avoidable lag, but the conclusion is the same :)

http://livingwithlag.com

Short answer, no.

As you've observed, even under theoretically perfect conditions, the latency is much more than would be tolerable for true real-time applications. And that's not even counting the time spent in hardware, on the various cpus in the path, buffers, etc.

What about a fragmentation of the area - what would be the radius from your location where you would be able to collaborate with others at <=10 milliseconds of latency?

http://www.timbercon.com/time-delay-of-light-in-fiber-calcul...

Roughly 2997 kilometres, with C as the speed.

What about with C' (being the current state-of-the-art/max)?

I guess the current speed, judging by [1] is about 200000 km/s (for light in optical fibre), which leads to 2000 km radius (or diameter, if more than 2 parties are involved).

[1] https://news.ycombinator.com/item?id=7862412

Yes, its' true. It turns out that the speed of light isn't really all that fast.

http://en.wikipedia.org/wiki/Superluminal_communication

The circumference of the earth is roughly 40 thousand kilometers.

Fun fact: This is easy to remember if you know that the kilometer was originally defined so that the distance from the equator to the poles was 10000 kilometers. https://en.wikipedia.org/wiki/Metre#Meridional_definition

Neat. So 10000km = an "arc" from the equator to a pole. If you assume the Earth is a sphere, the circumference at the equator = 4 * 10,000.

Yep, updated. Sorry for the typo.

67ms isn't too bad. First person shooter games feel pretty much fine at 67ms. Once you get to 100+ does it get very noticeable, and past 150 is when it gets unplayable.

However, it's important to note that FPS games use a lot of tricks with client-side estimations to infer the game state. But depending on the application, you could probably use similar tricks to make things feel responsive.

Yes, this can in theory be achieved through quantum entanglement: http://www.engadget.com/2014/05/30/physicists-claim-reliabil...

Nope. Quote from the article: "the process still requires the transmission of measurement data from one side to the other via classical communication (read: regular, non-instantaneous communication)".

See Stuart Cheshire's ""Latency and the Quest for Interactivity", from way back in 1996: http://www.stuartcheshire.org/papers/LatencyQuest.html

Check out ninjam, which tries to get around this by exploiting the idea that music "loops" in intervals anyway: https://www.youtube.com/watch?v=Gg2UtVdwQYo

I think the premise is a bit wrong.

When talking on the phone to someone on a satellite phone in a distant desert, you get a noticeable delay but its not a problem.

"Real time collaboration" anywhere on earth has worked for a century.

Through air you get 30ms latency after 10 meters (340 m/s * 30 ms). So, doing a back and forth communication is much harder.

Latency of what? Sound? Light? :)