Hacker News Reader

Comment by autocracy | Hacker News Reader

A side note: at least in python, I benchmarked MsgPack as 3 times faster than JSON, and a whopping 850 times faster to read than YAML. It seems unlikely that people will ever be editing this file by hand, so for unstructured data, especially where playlists can get very large, I suggest MsgPack over YAML.

BUT... you have a defined schema, so it's probably to your advantage to use a storage format with a defined schema: ProtoBuf or Thrift. That would mean somebody trying to use your code would already have generated objects in their language.

As for the hashing algorithm, this is a good use for MD5 -- cheap and fast. You're unlikely to be concerned about somebody actively trying to generate the same checksum for two music files. For non-security (integrity verification) purposes, MD5 is still very appropriate.

View on HN

Thank you, I agree. If I'm going to use JSON, I might as well use ProtoBuf.

About MD5, I was worried about a case where a service that serves user-submitted files would be exploited by MD5 collisions, leading users to open files that might exploit decoder bugs to execute code. Far-fetched, I know, but the tradeoff didn't seem worth it. I'm not married to that decision, though.

The question of hash usage made me think of an alternative approach -- what about some kind of audio perceptual hash? P-hash has support for audio hashes [1] (at least it claims to, but I've never used it). The metadata is useful, sure, but coupling it with the playlist seems like a bit of a strange design choice, if it could be avoided. In my mind, an ideal world would have two databases (or equivalent): one for metadata -> perceptual hash, and one for playlist -> List[perceptual hashes].

The downside of course is this requires pre-calculation of the p-hash for every track to use. But I can't think of a music application that doesn't require some kind of "library loading" step, so perhaps this could be accomplished then?

Of course none of this mitigates your concern with decoder bugs resulting in RCE, but I think that's probably best handled elsewhere (for example, sandboxed upload validation in your hypothetical user-uploaded-files service).

[1] http://www.phash.org/

The AcoustID fingerprint is exactly what you describe, and is already supported by the format. I agree that my crypto concerns are probably handled elsewhere, and, since I'm not actually doing crypto, I might as well add MD5/SHA1, which are more ubiquitous.

The most annoying thing is not dedicating the time to cultivate playlist with your favorite tracks, but to LOSE ALL of them, when you move or rename a file. And that is something that happens in 99,99% (I'd say) of all cases with any music library over time.

Example: I just moved all mp3, m4a files to the microSD card on my android phone, keeping names and folders identical, but my playlists are all empty now. Thank you Samsung! grr..

Making a path independent, p-hash independent (but utilizing, if available, or requested) playlist format is what I'd really want. The metadata should always be saved inside the file because metadata get's lost when you change the program. Filling up an sqlite db with all the metadata saved in your audio files would only speed up meta-data management and sync, but remove it's control from you.

Features I think make sense to expect from a perfect music player (without vendor-lock-in), be it run on mobile, web, desktop, cli or as a daemon:

• Path independant, file-name independant (thanks to p-hash) playlist import & export

• Playlist export options for ie. Samsung Music Player, iTunes or whatever crap we're locked-in currently

• Save Metadata incl. rating always inside audio files, because metadata get's lost when you change the program

• Extract Metdata from audio files into a database for speed, management and easier sync into the files

• Audio-Fingerprint may allow detection of: duplicates (hash independant), similars, classify genre and map mood-maps

• Batch-Convert between flac,mp3,ogg,mp4,m4a if user wishes without stupid dialogues. 128Kbit LAME-encoded MP3s don't sound converted to "best quality ogg"..

• Create P-Hashes (or another perceptual hash) initially, when idle, periodically or when requested

Example playlist format:

    estelle.mp3, audio-fingerprint, House-Playlist

    wu-tang.mp3, audio-fingerprint, PartyNight

You can apply a schema to JSON, too.

Can you hear us underneath that pile of bike sheds?

Haha, the bikeshedding argument is fair, but not terribly so. It's good to receive feedback of all sorts, and then the designer (me) thinks about it, distills it and makes something that's (hopefully) better than what was there before.

It's only bikeshedding when the decision has to be made by community!

Reply on news.ycombinator.com