Hacker News Reader

Comment by dahart | Hacker News Reader

I’m suddenly interested in making a neural network that can mute a known reference audio from a stream while leaving the surrounding audio in tact. Doesn’t something like this already exist? It seems like Instagram and YouTube could pretty easily leave audio-edited videos online while addressing the copyright violation, without resorting to takedowns or account bans.

View on HN

You don't need a neural network for this - after song identification, you might be able to stream the right song at the right location, then simply play the song "inverted" - you'll increase background noise but the song won't be identifiable.

It's a very simple DSP operation after song identification. It sounds like a lot of fun actually.

The front end location identification might not have an extant solution, but post identification it should be doable with standard tricks.

This is the cyberpunk future we were waiting for. Who wants to do some DSP?!

If anyone wants to work together on this, we absolutely should. This is a fun problem with a very, very cyberpunk vibe. Plus it's real DSP work!

That's what I was thinking too. It doesnt even have to be perfect. All it needs to do is mask/distort music to make the algo matching content id not identify it.

Hmmm.... would playing another song at the same time not have similar effect?

This might be fun weekend project :)

This won't magically make other audio sources that were present when the recording was made easier to hear, though. Once the recording is made, they're not independent any more. They'll still be masked, maybe even more.

No, but it will preserve them online for later filtering/post processing.

Maybe, depending on the details of your scheme, I guess? If you subtract data, that data is gone, and there's no separate stream of data in the recording for the cops' voices vs. the phone.

We just "play" the song - so it's a spotify call. Ideally anything not aligned with the song meant to induce takedown and the inverted playing of it will still be maintained.

To clarify, we don't want to subtract the data from the feed, we can just nullify it near the microphone.

I think it would be harder to filter on an instagram feed directly, not sure how their api looks or if it's possible?

I see, you're saying play the inverse at the recording time. So I misunderstood, I thought you were meaning doing it to the recording.

This is also complicated, because you need to factor in the relative locations of the original source, your source, and the pickup to make sure the phase is correct.

Level-wise they are separate though. It should be easy enough to find the proper gain for the reference recording (how loud does it sound in the video?), and then subtracted, leaving any residual energy in that band from desired sources. Yeah, there will be some gain-matching and spatial artifacts, but probably not enough to make things (voices for instance) unintelligible.

Thats actually where any ML component would naturally sit. It would try to basically determine a "close-enough" deconvolution envelope for the reference recording.

I don't think this is realistically achievable when you have all kinds of distortion and echoes. You'd also need a way to identify the exact position on the song and correct that if it was wrong because it was in the refrain.

I was thinking about this - a convolution filter with a long enough window would have a pretty good chance of identifying the right refrain location.

You're right about echoes, but it doesn't need to be perfect - the echoes are already probably below the detection floor for the takedown filter.

Also: If you identify and match up with a refrain, you'd still nullify that refrain. You'd just have to keep on your toes and move to the right location once you realized there was a mismatch. This is a classic cat/mouse "warring filters" problem!

Isn't this how balanced audio cables work effectively? Have two signals, one hot and one cold, invert them against each other and noise would be cancelled out. Replace noise with the song that got identified, and you can cancel it out.

I think if you time your destructive interference to the source of the audio perfectly and cancel out the sound, you haven't gotten the whole picture. As sound travels away from the source it reflects off surfaces and some of those reflections also reach the microphone. Those sounds have traveled a different distance than the original source and are no longer timed with your interfering element. Imagine holding your hand to block your direct view of a person in a house of mirrors but you can still see them in the mirrors. On top of that these reflections won't contain identical audio as the surfaces will absorb different spectral content, so now even if you had infinite perfectly timed anti-signals for each reflection, your anti-signal has spectral content that is missing from the incoming reflected audio and now your anti-signal is a signal unto itself. In order to create a signal that perfectly cancels out all of the copyrighted audio you would have to have knowledge about the form and materials of the physical environment as well as the position of the microphone within that environment.

This was already mentioned and answered to before here: https://news.ycombinator.com/item?id=26115933

In the end, the solution does not have to be perfect. Just have to hide the song well enough so the content ID cannot be made. As you say, the sound that will manage to bypass the "inverted balanced-like filter" will be distorted and hopefully content ID is not accurate enough to recognize it.

Yeah it does seem technically within reach and potentially solvable. I was thinking of including song identification when I mentioned an NN as a potential solution. I’m also assuming that there a bunch of other serious complications like resampling and clock skew between the two streams, a changing frequency response over time, etc. You can imagine that if there are audio dropouts or song pauses in the video you’re trying to fix, then naively subtracting the source song will result in accidentally playing the copyrighted song instead of removing it. And it might be trickier if people sing along to the song, etc. I think this situation is quite a bit tougher than the standard trick people use to remove ambient audio by recording with a 2nd mic further away and subtracting it.

Sounds like the Cocktail party problem.

https://www.comsol.com/blogs/have-you-heard-about-the-cockta...

It’s definitely similar, though should be a lot easier when one of the things you’re trying to distinguish and remove has a known high quality reference track you can use, right?

No the exact thing you're after, but you may be interested in Spleeter[0] which can pick out individual instruments from a given track with surprising accuracy. Deezer uses it to aid in song recognition. In this case, you could theoretically pick out the whole track and leave everything else or something.

[0] https://github.com/deezer/spleeter

Why doesn't YouTube do this to their videos, instead of muting all audio, or removing the video entirely? It'd be a far preferable user experience.

They don't get anything out of it except complaints when it doesn't work right.

Nah, I am more interested in charging the police officers with DMCA violations and penalties on top of disturbing the peace and purposeful interference with someone exercising their First Amendment rights.

Seriously, we don't need to fix the DMCA. This isn't the problem here. The problem is staring us in the face. Police are empowered by their unions which have a stranglehold on local if not national politics; even that 350 billion in the covid bills to shore up state finances is a little disguised bailout of public employee pension systems.

the simple fact is, public sector employee unions are a threat to the financial and physical security of all Americans. their actions permit their members to provide bad to no service in both schools and police if not general public services over all. Remember, qualified immunity is not just a police issue, it covers all public employees.

Isn’t undermining someone’s ability to use copyrighted material with the intent of causing a video takedown in the first place going to be a lot easier to achieve, and have the same outcome that you want?

It isn’t realistic to think the officers could be prosecuted for DMCA violations, nor is it even that clear cut - they aren’t posting the videos that are being taking down. It takes two parties here to result in the copyright violation.

This is definitely not a good look for law enforcement to fight against public videos this way, even if it’s a clever hack. It is potentially violating the law, and it runs against the very reasons they have body cams. But enforcement won’t be easy and requires proving intent, which makes it even harder. So much simpler to just take away this particular avenue for avoiding public records.

Reply on news.ycombinator.com