How practical is it to run some of this client side on the browser via WASM, with a few things offloaded to a server hit? Our computers are getting better. Think plug-in, think ad-blocker.
You're human being capable of understanding what you're seeing, ML is just computational statistics. You need to throw a ton of data at it to get accurate results.
You might be able to piggyback on pre-trained models via transfer learning in order to reduce the amount of data and resources needed to train a model for this task, though.
I don't think Shazam uses ML for song recognition, but instead hashing and fingerprinting via Fourier transforms.