JamRTC – WebRTC for Live Musicians

I'm getting <20ms with jamulus on a usb audio card on ubuntu studio (rt-linux). This is for a weekly rehearsal with a big-band. The server runs on an ODROID in the house of one of the players. Connection is with ethernet and glass fiber.

The experience is very nice.

My experiences playing with Jamulus during our lockdown follow.

Wired networking has less latency than wireless.

Sounds card and drivers add to the latency mix. Which is why Jamulus requires ASIO drivers (on windows at least) - the inbuilt mic on your laptop does a heap of processing to clean up etc, but adds to latency; unfortunately, if you remove this processing there's a lot of noise. A decent external sound card is pretty much required.

If you can isolate the sound of your instrument so you only hear it via the network (i.e. you're wearing isolating headphones), you can get away with more latency as your brain will compensate. That way, everyone will listening through the network, so everyone should be in sync (assuming the software compensates for different latencies). But this makes things really hard for vocalists or drummers, where isolation is really hard or impossible.

The upshot is that this is really hard for amateurs - I was trying to get something going with my community orchestra but gave up as half of them were on ipads or didn't know what an ethernet cable was, let alone an external sound card. Was tempted to look into building a raspberry-pi based standalone 'jamming' applicance, but then the lockdown ended and I lost interest. Still, exploring public Jamulus servers felt a bit like the early days of the Internet :)

We took a similar approach to ninjam but structured around a loop.

It's still in early stage but fun and has video.

https://www.pajam.live

How you gonna solve sync when the roundtrip adds a few hundred ms? Is the AI going to predict sounds 20 mins into the future after a while lol

While you’re at it, have the AI predict images from the NASA rovers

There's the speed of light on one hand and a maximum latency of roughly 30 ms, which is the boundary below which the human mind perceives sound as synchronous, on the other.

However, neither does prohibit non-colocated live music altogether.

I recall another project for this, Jacktrip, stating that people need to be within 500 miles of each other, all else being equal.

A marching band might span a whole American football field during a performance. That's potentially 135ms to the center of the field from the edge or more from a corner. Their type of music may be somewhat designed around it as a limitation as well as having a visual metronome in the conductor (though you could estimate one and even make it audible instead of visual using half round-trip-time for internet jam sessions).

People seated at near field level at opposite corners may hear a 350ms+ difference in some instruments, but they may spread them out so that it just sounds like a reverb/wash of echoes.

For these kind of apps wear headphones and you cut around 3ms per meter you are away from your speakers, due to speed of sound.

I've been jamming with some friends on Jamulus. A complicating factor is that the effect of latency depends on what instrument you play. When I play an electric instrument (bass guitar) that makes little or no sound of its own, then it's easy to monitor my timing in the mix through headphones, and my fingers just adapt by playing ahead of the beat as needed. Our bodies have to do that anyway in order to play in time because initiating a note happens before the note comes out of virtually any instrument.

It's much harder when you don't have that isolation, for instance when I play an acoustic instrument, I have to turn up my phones to drown out the acoustic sound. And it gets still harder if you hear that instrument by bone conduction, e.g., horns or voice. The band adapts by the "easy" instruments taking charge of the tempo and letting the other instruments float around a bit. Still, it takes a lot of concentration to avoid getting lost, and just doesn't sound as good as a live performance of a "tight" band.

The band members have agreed that it's better than not playing at all, but might not be a good listening experience for an audience. Not necessarily because the delays are perceptible but because it sounds like the band is struggling to stay together. Of course other things are happening as well, such as lack of visual cues.

Fortunately we're all "of a certain age," and between the weather warming up and us getting vaccinated, we'll be playing together again.

Sound travels only 3 meters in 10 ms so even bands playing in the same physical space have to deal with such latencies.