Hacker News Reader

Comment by crazygringo | Hacker News Reader

Most people have no idea of the amount of incredibly advanced signal processing that goes into echo cancellation and noise cancellation in videoconferencing.

This post is on noise cancellation specifically, and it actually has the potential to be a huge step forward.

One of the big audio problems with group meetings is that the background noise from each participant adds up, to a point where it quickly becomes unbearable. For that reason, videoconferencing generally only plays audio from one or two participants at most, using a fairly simple estimation of whichever audio signal is currently loudest. The problem is that this can make it really hard to interrupt (people will literally not hear you), or tell the difference between two people going "mm-hmm" versus the whole group. If you've ever been in a group meeting where everybody applauds something, this is why you see everyone applauding but only hear a smattering.

But if this noise cancellation really succeeds, it could be a huge leap forward because audio cues and overlap will actually work for the first time -- hearing the "mm-hmms", hearing everyone pipe up, and so on. Videoconferencing will feel more like an actual single shared audio environment, rather than the kind of "walkie-talkie" effect it so often feels like now.

I'm really looking forward to this.

View on HN

> The problem is that this can make it really hard to interrupt (people will literally not hear you)

This is driving me crazy with Google Meet in these COVID19 times. Even in a relatively small conference, I have a really hard time interrupting someone to ask a quick question, even when the speaker is expecting interruptions. It's always "excuse me!"; delay as person continues speaking; I stop; the other person says "yes, please ask away"; when I restart my question the other person already assumed I've changed my mind and continues speaking; repeat ad infinitum. And this is if they even hear me over the audio breaking up.

It's very, very frustrating. If they solve this it would hugely improve quality of life in remote conferencing for me.

If the speaker did hear you interrupt, then that's actually a latency issue, not a noise/mixing issue.

When a conference call is made up of people all in the same city on decent internet connections, latency is usually not a big issue.

But when a conference call has people from New York, San Francisco, and Japan on it, even if it's only 3 participants, latency can be bad just because of the speed of light, essentially (on top of what is otherwise reasonable hardware/software latency). Latency may be bad even if you're talking with a colleague in the same city, since the audio is "mixed" on the server, and that server might be across the world if a participant from across the world started the meeting. (Counterintuitively, the latency with your local colleague could be twice as bad as with the colleague from across the world.)

It's not just connection latency, it's decision making latency. As explained in the parent comment, all non-speaking mics are muted. If you want to interrupt, you have to make noise for long enough that it thinks you're talking now, before it will open up your mic. Otherwise it will think it's a cough or innocent tap on the desk.

You're probably right! Though I've just experienced this issue with three participants, all in the same city (not in the US though). It's really annoying.

This is also a problem of people not understanding that audio conferences aren't just a regular conference but with headphones.

There are a few things that most meetings could benefit from. Having an organizer who's aware of the differences between leading an in person meeting and a remote meeting, cutting video to save bandwidth if the meeting doesn't absolutely need it (the organizer can usually just disable the function), muting when you're not speaking (by far the best quality of life improvement, can be done silently by the organizer if someone is just doing their Vader impersonation throughout the meeting), using the "raise hand" function (again, the organizer plays a huge role here), using the native app instead of the web one usually provides better quality and performance, using a wired connection instead of wireless if possible, sometimes even starting meetings at non-standard hours (like 15 to/past the hour) helps avoid the rush of people logging in at the same time, etc.

I’ve experienced some pretty horrific latency in Google Meet that seemed to originate from my local device, where only my connection would suffer from high latency.

Typical restart-all-the-things usually made it go away. But it wasn’t unusual for 500ms of latency to slowly build up during a 30min call. Unfortunately I have nothing more useful to add, the issue resolved itself before I could track down a definitive cause.

Bluetooth.

Stop using it.

Without even looking at your setup, I would bet $100 minimum that it's Bluetooth latency. It adds a lot of latency, 500ms is not unusual, and many folks have no idea that all that latency is really just the last 18 inches. This is why you're seeing more and more cases where people are using good old iPhone wired earphone for conference calling, especially when skyping a TV interview.

> It adds a lot of latency, 500ms is not unusual…

How are you measuring? Half that is considered high-end from what I'm reading[1], and AirPods Pro apparently reduce that to 150ms.

[1] https://stephencoyle.net/airpods-pro

150ms is terrible. That's more time than going from London to Los Angeles and back.

Bluetooth definitely is the issue with 95% of these latency issues.

Also, video makes it 10-70 ms worse due to frame-buffering and significantly worse codec latency. If you're within 2-3h by car, try ethernet cables and Mumble/"Jitsii Meet", the former at a low-latency star point (locally, if you have to) and the latter in P2P mode. 30ms end2end is possible, but 50ms end2end is quite realistic if you have good internet routing. Inside a city even 10ms could be possible, but that'd need some ultra-low-latency software, no Coax/DSL (Ethernet-based FTTH should do), and a kinda-exotic low-latency soundcards, because Opus alone eats 5ms one-way in the restricted-celt-only-low-latency mode. This is basically the delay of sharing a round meeting table for ~20 persons.

150ms * 2 in many cases, as both parties use a bluetooth headset. Wifi adds another 5-50ms * 2 plus some packet drops.

Oh my god, I had no idea airpods were so bad.

Crazy that airpods have that much latency. I have bluetooth in the car and it has way too much latency - I thought Apple had improved it to not be noticeable but I never checked the numbers.

Incidentally, for playing midi instruments you generally want things to be below 8ms to feel natural. 150ms is an eternity.

Is the lipsync ok if you watch a video on an iphone with airpods?

> Is the lipsync ok if you watch a video on an iphone with AirPods?

For video they compensate for the latency by displaying the visuals slightly delayed (this has been done for over a decade even way back on feature phones).

Even game consoles have the option to do this since some TVs/receivers have audio or video latency.

Shure PGX-D is a prosumer / church-basement level of digital wireless microphone system, and that's 3.5ms.

I wish it was that simple. Unfortunately I was seeing this then using the laptops built in speakers, not just with Bluetooth headphones.

You can check your WiFi and try with an ethernet cable. Wifi has tendency to add unpredictable latency.

This applies to everyone on the call

> Even in a relatively small conference, I have a really hard time interrupting someone to ask a quick question, even when the speaker is expecting interruptions. It's always "excuse me!"; delay as person continues speaking; I stop; the other person says "yes, please ask away"; when I restart my question the other person already assumed I've changed my mind and continues speaking; repeat ad infinitum.

One way to solve this is to have the speaker name the person, and then wait until that person speaks. For example, if someone interrupts:

    Speaker: [Talks]
    Person A: Excuse me!
    Speaker: Yes, Mr. A? [waits]
    Person A: What about X?

Or if there are two people talking at the same time

    Speaker: [Talks]
    Person A and B: Excuse me!
    Speaker: Yes, Mr. A? Mr. B, I'll come back to you after A. [waits]
    Person A: What about X?
    Speaker: [Talk about X]. Mr B, you were saying?
    Person B: What about Y?

Treat it as a synchronization problem, with the speaker breaking the ties. As long as it's obvious to everyone whose turn it is to speak, it works well (assuming people aren't too rowdy/impolite).

Yeah, I wish there was a simple non-verbal option to signal intent-to-talk.

I want to just be able to hit my self-view and have it have a big icon on it or something so the person currently speaking (and everyone else) can see that I want to say something. Maybe sort these in chronological order so the speaker can see who wanted to talk first?).

In theory you could do this with a good chat, but for some reason the chat in Zoom and the others is kind of an afterthought and nobody uses it.

One of the reasons I prefer text based chat is multiple people can talk at the same time without needing to deal with interrupting audio. If you can type well, the bandwidth is higher for group communication (and you get a log).

At least with video you can kind of tell when someone is waiting to speak by seeing their expression. Audio only is worse (but maybe wouldn't be, if you had good intent-to-speak tools built into the app?)

I really like how Jitsi Meet puts the hand raising/lowering button right on the bottom bar, where there's just empty black/white space in Zoom/Google Meet, and not buried inside a menu labeled "Participants" (???), where it's a hassle to access (Zoom).

Microsoft Teams has a "raise your hand" function that's pretty handy.

>Yeah, I wish there was a simple non-verbal option to signal intent-to-talk.

this is a solved problem. webex has had a hand raise feature for a decade now. trivial for google to just copy.

>At least with video you can kind of tell when someone is waiting to speak by seeing their expression. Audio only is worse (but maybe wouldn't be, if you had good intent-to-speak tools built into the app?)

Which is one good reason to use video. At least with smaller meetings, someone can raise their hand or just look really pained. (Bigger meetings, you probably need to use chat.)

We rolled out https://chrome.google.com/webstore/detail/nod-reactions-for-... to all of our devices. The quick 'raise hand' button is great for what you're looking for.

I meet regularly with 2 people in the same room in one city, and me and one other person in the same room in another city (4 people, 2 locations). Google Meet is unfortunately entirely unusable for this purpose. It will select one person to "listen" to and tune out the person with the higher voice (that's me!). We have switched to Zoom for these meetings, as the only way to overcome this in Meet is to physically arrange for the higher-voiced people to sit immediately adjacent to a microphone -- while Zoom just works, most of the time.

The biggest reason for this is people not using headsets. If someone is just using their laptop speakers and mic, Meet will prioritize the mic if they're talking and will duck any audio that comes through the speakers.

So much this. I'm almost at the point of stating that echo cancellation has done more harm than good, because we are now in a situation where 80% of people have no idea that wearing earbuds could make a tremendous difference in call quality, and everyone just expects the software to magically take care of it.

Sadly, the software does not just magically take care of it. Anytime two people talk, a typical echo canceler just starts decimating frequencies until both of them are unintelligible.

Add in a couple of clueless teams who mount a camera/mic against a conference room wall and introduce massive amounts of room echo into the mix, and I'm at the point where a conference call becomes an absolutely mentally exhausting experience just trying to decipher what is being said. I have no hope of contributing, because I can only hear 2/3 of the syllables, and my brain is running on overdrive trying to turn those back into words. By the time I've figured out what they just said, they're half-way into the next sentence. What a stressful hellscape.

Ironically, if we had no echo cancellation, it would force everyone to use ear buds, and the average call quality would be a lot better.

100%. I wrote a post about this (among other basic tweaks) a couple of months ago: http://jonpurdy.com/2020/03/how-to-improve-your-zoomskype-te...

I have some screenshots of waveforms showing laptop mic vs headset, and the signal-to-noise ratio with the headset destroys even good noise-cancelling using a laptop mic that's farther away from one's mouth.

I have headsets and tried pretty much everything. there's always background noise from me. I'm even using the audio cable with my bluetooth (!) headsets, turning bluetooth off.

I don't know what else I can do.

There can still be background noise, but if you're wearing a headset there is not a feedback loop where noise from the other participants gets looped through your mic and speakers.

Participants often don't realize that they're the culprit when somebody else sounds terrible.

The software could require everyone to do a mic check before joining the meeting. It would record a second of them saying "hello" and play it back to them.

This would kill the product instantly. People don't like being reminded what their voice sounds like.

I use a Plantronics Legend bluetooth headset, which is pretty good at cutting out background noise. Tested with a phone.

Cheaper bluetooth headsets seem to pick up everything around them. Had that issue with a coworker where the headset was worse than using the internal mic.

Biggest and annoying issue though is consistent bluetooth disconnect/reconnect issues even on different MacOS machines. Latest firmware and such. Pretty sure it's not 2.4ghz interference.

I've heard that the original Bluetooth standard is pretty terrible for audio, especially for microphones. On Windows PCs at least, old protocols can cause a bad experience:

"Modern high-end Bluetooth headsets support AptX, an audio codec compression scheme that offers better sound quality. But AptX is only enabled if it’s supported on both the transmitter and receiver. When using a Bluetooth headset with a PC, it only works if your PC’s hardware and drivers are compatible." (https://www.howtogeek.com/354321/why-bluetooth-headsets-are-...)

Not sure if this applies to a Mac though.

It doesn't directly fix the disconnect/reconnect issue you're talking about, but I've found ToothFairy[1] super useful to mitigate the bulk of my MacOS bluetooth frustrations.

A few different ways it's come in handy for me:

- The Bluetooth speaker I use for music has a tendency to sporadically sound super hollow. Turns out it has a mic built in, and the voodoo of Mac's bluetooth stack would decide at random when it connected whether it would go into audio-only mode (and use the higher-bandwidth AAC codec) or go into audio+mic mode (splits the available bandwidth between the two and as a result uses a lower bitrate audio codec to compensate for the bandwidth drop). Used ToothFairy to always force that device into audio-only mode.

- After the above discovery, tested doing the same with my actual headset, and leveraging the built-in mic for input. For call audio, it's pretty erratic on whether it'll have any impact at all, and depends heavily on the circumstances of the call itself. Sometimes the audio is massively better, but most of the time the audio is already degraded when it gets to my machine and the bluetooth improvement is moot. That said, makes music in between calls far more pleasant.

- My bluetooth mouse is particularly susceptible to that consistent disconnect/reconnect issue you mentioned. ToothFairy can create a menubar icon for individual devices, which helps to act as a quick sanity check to see if my mouse has disconnected. ToothFairy can also run a shell script on disconnect, which has been handy. At this point I have it trigger a system notification[2] so I'm at least immediately made aware of it, check my idle time[3] in case it was the mouse going into sleep mode from inactivity, then conditionally leverage blueutil[4] to look for the device and reconnect if found (forcefully restarting the bluetooth stack in the process if it has issues). Doesn't fix whatever the root cause is for that consistent disconnect/reconnect issue, but this duct tape re-establishes a connection far more quickly when it happens, making the issue itself significantly less disruptive.

I only stumbled on it via Setapp[1] and tried it on a whim, but it's definitely one of the more handy utility apps I've found and well worth the $5 App Store price for anyone that has similar bluetooth frustrations with their Mac.

[1] https://c-command.com/toothfairy/

[2] https://code-maven.com/display-notification-from-the-mac-com...

[3] https://stackoverflow.com/a/17966890

[4] https://github.com/toy/blueutil

[5] https://setapp.com/apps/toothfairy

Some folks find that switching from WiFi to Ethernet for your home office can help:

https://www.jefftk.com/p/ethernet-is-worth-it-for-video-call...

I rolled my eyes at first, because the only client on my network was my streaming laptop.

Then I realized I'd moved to an apartment complex. I did a wifi scan, and found over fifty competing SSIDs.

Switched to ethernet, and the improvement was night and day.

It's easy to forget that only one wireless device on a channel can send at any given time, even if they're on different networks. And that every channel overlaps with its neighbors.

100% agree. Latency is a killer to natural conversation so if you want to be your best on a conference call, no Wi-Fi and no Bluetooth.

I always feel this with Zoom. Interestingly it did not seem to be an issue on a recent Discord call.

Discord targets gaming which absolutely prioritizes low-latency. Zoom has a very noticeable amount of latency which makes it really awkward to have multiple people talking at the same time.

Does anyone have more details on Zoom vs. Discord latency? We've been experimenting with Zoom and Discord for online trivia tournaments where if one participant had better latency than another that would give a big advantage. I'm sure that has to happen on any platform, but if there's a bigger variance on one platform vs. the other that would be good to know.

If you read trough other comment threads, you can learn all kinds of things adding/reducing latency.

No matter which software you use, some people will be at an advantage simply due to their isp/wired connection/wired mic.

This could be a business idea though: Conferencing software which equalizes everyone’s latency.

We're trying karaoke through Zoom tomorrow as a standup gag, wonder how that will work with the delay haha.

my perception with zoom (only based use, not actual knowledge of how it works) is that it has two modes: one where it tries to isolate the speaker and auto-mute everybody else, and another where it can't figure out who the speaker is and just lets all audio through. so if everybody on the call is singing together, it should all come through.

Is Zoom picking some other point on an optimization curve, and if so, what's more important to it?

Or is it just worse?

Zoom seems to be optimizing for bandwidth use, and by extension, cost to them. Its typical use case is a shared office internet connection.

Discord users are more likely to have a dedicated fast internet connection and doesn't seem to care about profitability at the moment.

It's just the difference in designing for a 100/10 connection to yourself vs sharing a 100/100 connection with 20 other people. Zoom reasonably gracefully degrades on choppy/slow connections while Discord becomes straight up unusable.

Ah, this makes sense. I have noticed that Zoom never quite gives up for participants on slow / bursty networks. When something becomes that bad I expect it to be headed towards failure, but Zoom is happy to just sit there at 0.5fps.

It would be nice if there was a 'raise your hand' button which put you in a queue to speak. Even better if it let you take a quick note in case you forget what you wanted to say.

I think part of the problem is that the tooling and the societal norms still need to evolve. The tooling is getting there - Zoom/Teams (I don't know about Meets) have buttons to communicate out-of-band beyond just text chat. We need to have more of that, I imagine eventually we'll have a wide range of ways to express ourselves (and customs/norms to match). Although I don't know if that'll happen before most people stop working from home.

Very true. At the moment the best workaround seems to be Microsoft Teams allowing you to "put your hand up" through a button press so that the speaker can give way to you, but this is far from intuitive/comfortable for most people compared to how they'd normally interact.

Even in on location meetings I ask people not to interrupt and raise their hands instead. Otherwise a few socially inept people keep interrupting to ask questions or bring up criticism that the speaker would have addressed of they had been allowed to keep talking.

That sounds like network latency, not an issue with switching audio sources.

Very likely. But for whatever reason, it results in very frustrating meetings. Like someone else mentioned, Discord seems to work better -- even though it's not the same use case, and I use Discord with friends and much later during the day.

Well, you think they're expecting interruptions. ;) Maybe the existing Meet is good for you.

I know they are expecting interruptions because sometimes (as in the particular example that triggered my frustration) they said so at the beginning: "please interrupt me every time you have a question, otherwise I'll feel like I'm speaking into the void and this will be a very boring meeting".

The problem is that Google Meet (or my connection, or whatever technical reason) wasn't up to the task. This has happened enough times that I dread interrupting now. Sadly, one person monologuing is not how face-to-face meetings really work.

I wish everyone would just get a headset. It drives me nuts when people call me on speakerphone, of course they never hear the problem.

Tell them you can't hear them due to noise, tell them to pick up the phone or plug in a headset, etc.

They will probably tell you everyone else hears them, so the problem is on your end.

People just don’t care about quality, they will gladly use the crappiest mic with all the noise all day..

I mean they can tell whatever they want. I organized my work around the fact that I can tell people to go fuck themselves when they are uncooperative in these small but very important ways. Obviously that's not the reality for ~90% of people today.

But people shouldn't take "workspace" abuse from others. Be polite, assertive, constructive. Give them credit for taking the time to care a bit about this problem, offer to debug with them later, try to demonstrate it - switch to speaker, capture what you hear. Alternatively just switch off voice and type. Tell them that the sound is garbage for you so you switched for typing. Try to do quick 1-on-1s instead of the group call, even offer to write a summary after.

This is basically the equivalent of constantly not giving a fuck about how loud one is in an open office. There ought to be proper channels [ha, no pun intended] to address and solve these.