Creating Shazam in Java (2010)

Around the time ML/AI was starting to take off again, (I want to say 2011/2012 ish) I was doing research into applying OCR methods to recognise engineering symbols on drawings (both hand-drawn as a graphical rep and CAD as a vector rep). OCR uses locally-sensitive-hashing techniques, where minute changes, like a couple of pixels difference here and there, in theory result in similar hashes. What you really need to do is look at your windowing and overlaps and tune those to get something that actually gives you localities that are useful.

This worked quite well for me. After running a "training-set" of sorts I created a small tool that ran over a quarter-million engineering drawings to get counts of each symbol from the set. (I'm going to hand-wave some implementation complexity here but essentially) If an exact match couldn't be found, the item being searched would show with thumbnails of it's nearest neighbours off to the side, you could select if it was essentially the same as one of those. (Sort of like "is this, this person" in Google Photos and the Apple equivalent)

The next step after this POC was to expand to discover symbols used most next to other symbols to use as contextual menu items in other CAD software to speed up drawing production, since a lot of time was spent placing a symbol, then stopping to text search the item you were placing next.

Unfortunately I was retrenched soon after and didn't get to progress. A couple of years later I took this a dimension further and was prepping for a PHD that would look at this for 3D-objects and models using naively generated voxel representations. Almost by accident I found group at/backed-by DARPA had a patent pending on a similar method. In retrospect I should have just gone all in on a photogrammetry based method since that kind of won out as a superior method - but it was still early days.

The moral of the story is that everything is pattern recognition and simple methods from before this last decade of ML/AI could do some cool stuff too.

that's basically what it does, it finds peaks relative to total power.

you can think of it as like identifying continents by constellations of mountain tops which is robust to whatever chaos might be happening at sea level.

I wonder if they have such pattern in EU or Netherlands. US tends to patent stuff that wouldn't fly in Europe.

The purpose of a patent is to encourage global innovation by granting exclusive rights for commercialising technology for a limited time in response to sharing how it works.

Roughly, if you invent PageRank and keep it internal to your search company, you can exploit that idea but if someone else comes up with it they can exploit it too. If you invent PageRank and patent it, the whole world knows how it works but can’t use it for 20 years without paying you.

Generally in case of a violation you’d want to sue for damages, whether that be licensing costs not paid, or missed revenue on your own side.

This probably then ends up in a bit of a grey area. Shazam would be within their rights to exert the patent, but the damages are likely so minimal to be outweighed by the cost of litigating the case, which means that the C&D is a little too chilling for my liking. Basically, any penalty would be lawyers fees - author hasn’t tried to commercialise it, it doesn’t reduce Shazam’s revenue at all. Are there any damages at all?

And then, few would argue (I think) that simply implementing a patent for educational purposes constitutes infringement. My uni course contained an implementation of PageRank. Is publishing the course notes then infringement? Arguable it’s anti-goal to discourage actually spending time learning about the innovation beyond just reading the patent doc - otherwise the motivations of the system break down. So, just how different is this? Presumably on a conceptual basis you want your universities to be able to teach about things invented in the last 20 years without licensing patents?

Why do people comment on things like patents when they don't know anything about them?

The etymology is from "letters patent", i.e. a public description of how the invention works. The social contract here is that the inventor gives up the details to the public, in exchange for temporary exclusive use rights granted by the public.

It means that, instead of incremental improvements on an idea, these can now be limited to 20 years per step.

> robust to noise but not warping in time or frequency domains.

Could you use something like a dynamic time warping algorithm for this? (I'm not super acquainted with the technique and not sure if you could get away with it in the frequency domain used for the matching.)