Hacker News Reader

Comment by crazygringo | Hacker News Reader

> to turn every song into a hash

Just to be clear, it's not turning each song into a hash.

It's turning each song into many hundreds (thousands?) of hashes.

And then you're looking for the greatest number of mostly-consecutive matches of tens (or low hundreds) of hashes from your shorter sample.

Also, I don't think this would be done with training a model today, because you're adding many, many new songs each day, that would necessitate constant retraining. Hashes are still going to be the superior approach, not just for efficiency but for robustness generally.

View on HN

I'm an MLE, I would probably chop the songs into short segments, add noise (particularly trying to layer in people talking, room noise, and apply frequency-based filtering), and create a dataset like that. Then I would create contrastive embeddings with a hinge loss with a convnet on the spectrogram.

Ultimately this looks the same, but the "hashes" come from a convnet now. But you still are doing some nearest neighbor thing to actually choose the best match.

I imagine this is what 90% of MLEs would do, not sure if it would work better or worse than what Shazaam did. Prior to knowing Shazaam works, I might think this is a pretty hard problem, knowing Shazaam works, I am very confident the approach above would be competitive.

Why add noise to the training set, rather than attempt to denoise the input?

To start from an original song and move it towards something that resebme a real life recording ? IOW : make the NN learn to distinguish between the song sound and its environment ?

I'd assume adding noise is done once per song and is thus a bit computationally cheaper than trying to denoise each input.

Because then you’re training it on data that is more similar to the operating environment for the application. It’s a better fit for purpose. If the target environment was a clean audio signal, you’d optimise for that instead.

So you want a location-sensitive hash, or embedding, and you want it to be noise resistant.

The ml approach is to define a family of data augmentations A, and a network N, such that for some augmentation f, we have N(f(x)) ~= N(x). Then we learn the weights of N, and on real data have N(x')~=N(x).

The denoising approach is to define a set of denoising algorithms D and hash function H, so that H(D(x'))~=H(x). This largely relies on D(x')~=x, which may have real problems.

So the neutral network learns the function we actually need, with the properties we want, where the denoiser is designed for a proxy problem.

But that's not all...

Eventually our noise model needs extending (eg, reverb is a problem): the ML approach adds a new set of augmentations to A. This is fine: it's easy to add new augmentations.

But the denoiser might need some real algorithm work, and hope that there's no bad interaction with other parts of the pipeline, or too much additional compute overhead. (And de-reverb is notoriously hard.)

Adding noise is generally helpful for regularization in ML. Most modern deep learning approaches do this in one way or the other - mostly dropout. It improves generalization capabilities of the model.

That could work, I think denoising a song to be a perfect match to the original recording is probably a very hard problem, so hard that your model will still need to be robust to some deviation from the original track, and therefore you need to do what I said above anyway.

Generally it's much easier to generate noised pairs from clean input than it is to do the reverse, i.e. go record lots of noised inputs from the wild and match to the original song. So the denoising problem you mention would be tougher still due to covariate shift. I think the features you learn trying to fingerprint the song through noise will probably be a bit more robust, but I don't have a mathematical proof.

Depends what the model targets. If I gave this problem to a bunch of musicians, they’d be pulling out features like the key, tempo, meter, chord progressions, any distinctive riffs or basslines, etc. Those are the things hashes could be built from and would be more information-dense than samples of the particular recording.

Using a model to deconstruct a song like that might enable the ability to recognize someone playing the opening bars of Mr. Brightside on a piano in a loud bar as well as its drunkest patrons.

you couldn't recognize anything ambient like, let's say, Loscil

Hard times for Indian classical music, as well...

"Features" was Pandora's original design (Music Genome Project). IIRC you can still see it in the UI in how they describe songs.

> that would necessitate constant retraining

You wouldn't necessarily need to retrain that frequently. If your model outputs hashes / vectors that can be used for searching, you just need to run inference on your new data as it comes in.

Definitely this, embeddings would be the modern approach.

The "modern" approach sounds like it would be a lot worse than this approach both in terms of input and runtime performance.

Trendy ("modern") is not necessarily better.

You would probably take the approach that is used for face recognition. You train a model that can tell if two faces are the “same”. Then you match an unknown face against your database of faces. You can get clever and pull out the “encoding” of a face from the trained model.

You would just rename hashes to embeddings.

Reply on news.ycombinator.com