Horrible edge cases when dealing with music
dustri.org
dustri.org
The Apple Music style guide is a great and accessible place to look if you're interested in music metadata: https://help.apple.com/itc/musicstyleguide/en.lproj/static.h...
DDEX is the set of metadata standards used across the music industry: https://ddex.net/
Also worth knowing that not all content on streaming services is music! Some of it is spoken word, ASMR, non-music field recordings, etc. The difference between sound and music is subjective of course though.
On an artistic note, music can of course be presented in an infinite amount of ways. Not all these can be represented (i.e., re-presented) on streaming platforms. Installations and generative music for example. That's ok! To be able to represent all this music (and non-music) requires restrictions, otherwise we wouldn't be able to create programs for it in the first place.
Sometimes restrictions on sound content are to make the streaming services friendlier to use, like the formatting of featured artists in titles. Royalty laws put restrictions on the handling of different roles, like songwriters and composers.
BUT nothing is stopping you from building your own way of representing music digitally, so long as you follow relevant laws.
For a proper attempt to preserve metadata, I'd refer to the MusicBrainz Style Guideline — https://musicbrainz.org/doc/Style
Is there some sort of rule against sirens?
The song "Car Alarm" prominently features a car alarm on Youtube, but not on Spotify. This could just be a choice by the artists though, rather than a platform restriction.
https://www.youtube.com/watch?v=xV7nHX2RLjQ
https://open.spotify.com/album/2u4HDb57v96iiJZUC7PqOx?highli...
Here's the regular version, same as Spotify, also on Youtube: https://www.youtube.com/watch?v=N4inWiKCt-E
I don't know if Spotify has music videos, but Apple Music for example does, so you can have both versions.
Even some albums that are clearly about music can contain non-music, take Tenacious D's skits as an example.
The albums Ангедония (https://open.spotify.com/album/1HxmgR8wpc1ySplYCTNwaW or https://i.imgur.com/ndOXsgL.png) and Продано! (https://open.spotify.com/album/5kp7j9B4TDA3VfhaXz9XcJ or https://i.imgur.com/QJNjXc1.png), for example, both contain a version of the track "Рижская". The former uses the original Russian while the latter uses Rigas', which is a sort-of translation (it's an adjectival version of "Riga", which doesn't really exist in English).
https://help.apple.com/itc/musicstyleguide/en.lproj/static.h... indicates that there are separate fields for each of the original, the translation, and the transliteration, but it doesn't seem like anyone actually _uses_ these, and instead just picks one arbitrarily to stuff into the main/native field. I'm not sure if Spotify even has a way to display the alternates--I can't recall ever seeing a toggle for it, though idk if I've ever actually encountered a track that includes the variants, since you can't see raw metadata on Spotify.
Is it just that music labels do a lackluster job of handling metadata? I expect this is probably the case, since classical music is similarly messy--it's a crapshoot whether the composer or performer gets used as the artist. For that I've at least seen the composer metadata field populated sometimes, but internationalization fields don't seem to be used ever.
I'm not saying this is necessarily a good design decision, just pointing out what might motivate this design
I also usually want to know who the conductor was and what was the orchestra
Unfortunately, we couldn't make it work economically even though we tried several approaches for monetization and had to shutdown a while ago.
Are there any legal reasons preventing this ? I don't have the ability to play classical music myself, ( my weighted 88 key sits taunting me), but I'd love to be able to take these compositions and integrate them into my own music.
As always, this isn’t universal. Check your local copyright laws to be sure.
(E.g. My handwritten copy of a PD short story is probably PD. But my reading of it isn't.)
BUT if some famous piano player performed for the MIDI recording, then you've got tempo changes, timing, attack velocities, all that data in the file as well. Probably not PD (best to ask). (As with piano-player rolls.)
You have the rare hip hop song which samples classical music directly, but I really want to learn music theory, like understanding how the notes work.
It's been a long while for me, but a search on 'classic midi files' turned this up. (Hard to tell if it's crap without a listen.) https://www.midiworld.com/classic.htm
Or maybe share some tips on what works with the corpus of classical metadata?
All team members have started new opportunities since quite a while and untangling all this, filtering out potentially copyrighted/licensed material etc. would take quite some effort.
Music metadata has been optimized to suit the marketing preferences of contemporary record labels. But more general classes of music won't shift units on Spotify so the message is loud and clear, a resounding "get fucked" from the assholes that run this industry.
(One side of my family is in the recording business. I can confirm the whole thing is run by assholes.)
Which one ? Played by Kissin ? Or by Rubinstein ? Or by ...
Artists/conductors 'perform' 'interpretations' of works. Whether any interpretation is 'masterful' or 'virtuosic', etc., is up to the critical listener.
Three recordings by Horowitz may have been released ... which one is 'best'? Depends on the phase of the moon and your playback equipment!
The 'quality' of a particular performance may depend on the instrument provided to the artist. The year of recording can be very important (reflects techology used). The acoustics of the recording venue are also inherently part of the performance ... e.g. too much reverb.
I. Adagio sostenuto
II. Allegretto
III. Presto agitato
https://musicbrainz.org/doc/MusicBrainz_Database/Schema
10+ years ago at Zvooq, during the golden age of music metadata and recsys startups, we tried to really solve entity resolution in the domain. I've never seen another streaming service make an honest attempt at this, even at Spotify with their 2014 acquihire of the Echo Nest. You still get Utada and Hikaru Utada as ~completely separate entities
The schema is here in case you are curious: https://github.com/WhatCD/Gazelle/blob/master/gazelle.sql
(btw., yes, I bought a large percentage and, nowadays, mostly buy on bandcamp [2]).
I'm still technically an automod although I haven't participated for a long time.
A place this diversity crops up most commonly is the so called "standard" firstname middlename lastname fields. This just doesn't work in India where people may have even 7 components to their names ... and who are you, dear system developer, to dictate what their name should be like.
The name structure diversity I understand is a problem for Europe too with hyphenated names. This brilliant sketch by Fry and Laurie says it all - https://youtu.be/1LopIroSjsU
There were four bands called Kaleidoscope, all active around the same time: one from England, one from Mexico, one from the USA, and one from Canada. To disambiguate them, generally the country is specified, e.g. Kaleidoscope (UK).
There were two bands called Bulldog Breed, both of which recorded albums called Made in England. They're decades apart, but as albums can be re-released, they're normally disambiguated by record label, e.g. Bulldog Breed: Made in England (Deram).
There are two Nirvanas, the original UK-based band, and the US one. The UK Nirvana recorded a cover of Lithium.
You'll be aware that the Beatles recorded an album called Revolver. A Russian band, called Revolver, recorded an album called Beatles.
Do you include the definite article in the band's name? If so, what about The The?
Tintern Abbey released a single single, the A-side of which was called Beeside.
Fortunately, I lack any amount of musical talent, so this will never happen.
[0] I know, REM beat me to it.
My SiriusXM radio omits leading articles like “The”, so “The Beatles” is displayed as “Beatles”. I alway long wondered how it would display “The The”. Would it be “The”, empty string “”, “The The”, or something else entirely. I eventually learned the answer: “The The”. Which is good, but then made me frustrated that it wouldn’t allow the “The Beatles” to be “The Beatles”.
Example album: https://drumkoon.bandcamp.com/album/they-tried-to-ban-this
He has hits like "Hey Gugle Play Music" and "Hey Siri Play Space Music".
And some times they're altered without any obviously-visible indication like differing number of tracks or "edition" naming, e.g.:
- MM..FOOD? (2004) [RSE0051-2] where "Kookies" samples Sesame Street: https://www.youtube.com/watch?v=4RYCLfGE-_Q
- MM..FOOD? (2007) [RSE0084-2] where "Kookies" is totally re-recorded with a different and non-infringing beat: https://www.youtube.com/watch?v=8iYSwvdEfeY
If you use a streaming service you're hearing the latter and might not ever realize the former existed. The re-record is a fine song, but the original fits much better with the rest of the album.
I'll sign up for the streaming service that provides the original 1990 Rust in Peace.
I'd love a version of Killing is my Business with the remastered content were it is identical to the original, but with none of the content or artistic changes. Something like the despecialized Star Wars videos.
OK, found it! It seems that the desirable keyword is "Reissue" and specifically to avoid the keyword "Remaster". There was at least a Reissued but not Remastered LP in 2013. Maybe more, but that javascript is horrible!
And maybe I should feel like it's a luxury to have extra bonus tracks with more takes at the end - but I lean over to the album purity side more - the album is one artwork by itself, I want the original track listing and nothing more, as "the album".
On a number of live Bob Dylan albums, for inexplicable reasons all online versions (not just streaming, but the download releaes, too) are now missing the between-tracks content that on the physical CDs is part of the pregap, which means you're missing out on the funny stage banter on the 1964 Halloween concert, or the infamous heckling on the electric half of the 1966 Manchester concert.
[0] https://www.discogs.com/release/8369169-Ozma-Songs-Of-Audibl...
[1] https://www.discogs.com/release/15809071-Ozma-Songs-Of-Inaud...
This has caused last.fm to think "Fun" is one of my most played songs of the year even though they were all separate tracks.
Here's one of the best albums by Hype Williams the duo: https://www.discogs.com/master/337242-Hype-Williams-Untitled (not edm but still).
Moreover, last.fm also has an issue with unicode characters and some artists/albums may have two or more different profiles. It's a mess!
Anyway, personally I long since abandoned Last.fm in favor of more hands-on discovery.
Fun fact: he adopted it as his stage name to spite Warner Brothers Music, due to the unfavorable contract terms and the creative meddling they kept doing (among other things). By terms of the contract they had to go along with whatever stage name he picked. They had to spend a bunch of money having a custom font created, then mailed on floppy disk to every music reviewer, newspaper, magazine, radio station, etc as part of album promotion. No one was on the internet in those days so it ended up causing them a lot of trouble. I guess he found his own loophole in the contract.
Take "artist". An album can have a compilation of tracks, each from a different artist. A given track can have multiple artists, eg a guest singer. There can be multiple composers of a track, there can be multiple writers of lyrics, and there are usually several performers (even if they all are part of the same band).
An album can have multiple names, or no name at all (which often means people refer to it in different ways, giving it multiple effective names). Likewise with tracks.
Albums and tracks get re-released, sometimes re-mastered, sometimes re-recorded entirely, without the names changing.
The same recording of a given track might be released on multiple albums, eg as a single in a pre-release and then within the main album.
Tracks might be mastered differently for different media (Radio vs CD vs background use in cinema, etc).
Names of any of these things don't have to be representable in any existing text encoding, or even any combination of encodings.
Artist name should be a list (or other multiple element structure).
Track name should be a list.
Album name should be a list.
Etc. Assume a malicious artist will find a way to make the assumption that any field has a unique value incorrect.
Right. It's a relational database with lots of many-to-many relationships. We need to stop trying to make it a flat file. A FOSS SQLite schema that everyone can embed, or even a turnkey little db, would be wonderful.
They really want to fit everything into the artist/album/song scheme, and that doesn't really work well for much classical music. In classical music most of us would want something like composer/piece/orchestra/conductor/performers (the "performers" field would be for performers in addition to the normal orchestra, such as soloists or choruses), and it should be possible to change the sort key order among those fields.
I think your list is pretty good (composer/piece/orchestra/conductor/performers). It'd be nice to also have a field for the movement. Also I totally agree with custom sort order being a must. E.g. for choir pieces swap out 'orchestra' for 'choir', for piano pieces only include 'performer' field, concertos should have the 'soloist' before the orchestra, etc.
Apple will apparently be solving this problem this year with a dedicated "Apple Classical" app, the origin of which is their Primephonic (https://www.primephonic.com/) acquihire.
"Why are you shutting down Primephonic?"
"To focus on creating an even better experience for customers around the world. Apple Music plans to launch a dedicated classical music app next year combining Primephonic’s classical user interface that fans have grown to love with more added features."
One missing feature that I've never seen in a music library application is the ability to have "presets" for these column headers that you can easily switch between. So you can turn on the "classical" preset which shows Composer, Performer, etc., and you can turn on the "contemporary" preset which shows Artist and Album.
It’s one of my favorite pieces of software, a true music library manager and not a music player with some management features tacked on.
I used to be quite happy with a local music library and playing it with mpd, but at some point I got sick of maintaining over 100 GB of music library across multiple devices. The streaming solutions for mpd are quite awkward to use, otherwise I would have stuck with it just to get the composer field.
I uploaded several pieces from Swan Lake, and listed each of them as the particular piece that they were. For example, https://musescore.com/user/36584999/scores/6642212 is Swan Lake, No. 4 (Pas de trois), part IV.
But every time I list a piece like this, it gets "corrected" to the canonical reference, which is "Swan Lake". And the "other scores based on the same work" are totally useless, because they can be any piece from anywhere in the ballet. It's more than 600 pages long! Where's the value add from removing my indication of exactly which piece of Swan Lake I just scored?
Per track.
Plus artwork. With credits.
Plus random thank-yous and special thank-yous to sponsors and other contributors.
The importance of each field varies with each project. For classical conductor showcase projects the conductor is the headliner. For classical composers, obviously it's the composer. For compilations it can be a concept like "Classical Chill" or "Early Music from the 15th Century." For pop projects it's usually the band/artist name. For movie tie-ins, it's the movie, or perhaps the franchise.
And so on.
You get all of this for free on CD/vinyl labels, and the fact that it's freeform doesn't matter because the sleeve artwork and visual placing of the elements make it clear who/what matters.
Trying to fit this into a single fixed metadata schema is madness. It can't be done.
This is one of those situations where trying to data-fy the information leads to huge loss of detail. You get some nominal (poor) searchability, but you give up a huge amount of interesting supporting information.
What the industry needs is a digital sleeve note standard which can handle all of this and more.
But metadata is designed for sales and stream tracking, not for user satisfaction. Which is why it's so bad at capturing what users want.
Why? I'm not sure about US law, but here in Poland it would definitely still be copyrighted (even if you'd be effectively granting an automatic implicit license to use that name for the purpose of referencing the work).
I much prefer this informative post (with a clear title and plenty of examples and explanations) to the uncommented "Falsehoods programmers believe about ..." listicles.
p.s. that video was awesome. Were those 8" floppies it used?
But those specialized music workstations all were custom designed so the parts etc. used were based on older standards I’m sure. Pre-ISA bus too I would guess.
Nowadays, even a lot of embedded devices like IOT things are running effectively a Linux OS on them so the economies of scale are enormous compared to the early 80s music gear.
I imagine Roland and other similar companies had their own OS or base platform they used on those earlier workstations that eventually had screens you could attach.
Interested about why a significant number of albums have Japanese promo versions with extra tracks. Is that a cultural thing about Japan or is there another specific reason? (I know it's not only Japan but mostly Japanese editions that I see throughout the web)
As if, almost even brouillard has a Japanese bonus track named brouillard in their album brouillard.
https://www.wsj.com/articles/a-gift-for-music-lovers-who-hav...
[*] Actually their style of music is unique enough that it's given rise to its own genre, "Zeuhl".
You wouldn’t index a collection of records by the shape of its sonic wave forms, would you?
No more would you index by the artistic representations on the cover which, as we have seen, includes everything including the words purporting to be the artist name and the title of the release.
No, of course not! You use the barcode on the back!
https://en.m.wikipedia.org/wiki/Catalog_number_(music)
For white labels, I guess you have to hide them in a pile until they get a release.
I recently started a fun/side project which aims to improve my personal tedious process of (a) managing a queue of things I want to listen to, (b) keeping track of which of them I listened/liked and (c) easily sharing my "discovery activity" with my friends who share the same passion.
The idea is that you can add items to a "Queue" list, using Discogs Master URLs (e.g. https://www.discogs.com/master/1994809-Alt%C4%B1n-G%C3%BCn-Y...) or Spotify album links, because this is what I use in my day to day.
So I've started by parsing Discogs' data dumps[1] to backfill my database (aka. "release groups" in MusicBrainz), so that I avoid hitting Discogs API with every user query) with all "master" releases. But then realized that (a) they don't include cover images and (b) not every release has a corresponding "master" release and (c) they're not updated frequently enough (maybe monthly).
Then I thought of using MusicBrainz data dumps[2] to augment the entries by Discogs in my database, which do provide cover art images, but then how do I correlate them with the already-inserted Discogs releases? Fortunately there's a Discogs URL for many release groups from MusicBrainz; unfortunately only 50% of them do have it.
Then I could perhaps use solely MusicBrainz data and not Discogs at all, but then what if people are mostly using Discogs links? This will result in two identical records on my database, pointing to the same underlying release but in different sites (Discogs & MusicBrainz). Perhaps this can only be solved by human moderation and providing the ability to "merge two items".
Then I thought of actually contributing to MusicBrainz and add the Discogs URL for every release group.
Another interesting assumption I made, which turned out incorrect, was the every [artist, release title] pair was unique. But that's not the case - an artist may have multiple "master" releases (i.e. completely different tracks inside) with the same title.
And, yeah, if I was still collecting my music as CDs? Keygen Church would be filed next to the MBR because of that, not over in the Ks.
There are two well-known Avishai Cohens, both jazz musicians (so you can't disambiguate on the broader genre), both from Israel (so you can't disambiguate on country of origin), but luckily they do play different instruments (bass and trumpet).
---
The Australian post-rock/shoegazeish band The Dead Sea https://www.discogs.com/artist/1450625-The-Dead-Sea released a self-titled EP with 3 tracks in 2006.
2009 they released a full self-titled album (only 1 song from the EP featured on it).
I can't even find the "second" release on discogs.
Both were self-released, so no way to disambiguate on ISMN or record label/publisher.
So you'd have to disambiguate only on the year of release, or define what demarcates an EP from a full album and mark them as such (there are some competing definitions out there, since it became less clear after the introduction of the CD, and now digital, but even back then you had complections like maxi singles and mini-LPs, IIRC).
---
Many of John Zorn's releases tend to be headaches:
"Tap"
Composed and produced by: John Zorn
Performed by: Pat Metheny
Subtitle "Book of Angels, vol. 20"
Overarching series "Masada Book 2"
Released on Tzadik (Zorn's personal label) as a John Zorn/Pat Metheny output.
Licensed to Nonesuch (Metheny's label) as a Pat Metheny release with different cover.
You can fill a large closet collecting just his stuff so I'm not sure why digital is off the table.
Along very similar lines, there are also famously two musicians named Bill Evans. Both play jazz (genre) and both are from the US (country). As a bonus, both have been members of bands led by Miles Davis. And again you can differentiate them by instrument (piano and saxophone).
Which is sort of what Wikipedia article titles do to differentiate them:
https://en.wikipedia.org/wiki/Bill_Evans
https://en.wikipedia.org/wiki/Bill_Evans_%28saxophonist%29
I guess Wikipedia also differentiates by level of fame, since the pianist article title doesn't have a qualifier.
> † from Justice
> from David Bowie
> ' from Frank Zappa
> () from Sigur Rós
How are these albums names "edge cases"? I feel like ANY software would deal with them just fine. Hell, the last two don't even use non-ascii characters (though you definitely should have unicode support regardless).
Edit: I just realize HN won't the star symbol (U+2605). Which is weird since it does support characters like Chinese/Japanese glyphs あいうえお你好
> Edit: I just realize HN can't display [star]
It doesn't do emojis and various graphic characters. This is supposedly a "feature" shrug-emoji-here
album='
albumsort=Apostrophe
These can act as search input as well, so searching for 'apostrophe' could yield it depending on the software used.† can either just sort wherever Unicode puts it, or if the album has a known pronunciation like 'star' above, have a sorting key too.
Sure, but it doesn't happen automatically, so it's not "just do [..]". You need to think about some of these edge cases.
I wrote a simple music library/player a bunch of years ago and ran in to some of these issues. I never did figure out how to deal with the band named "" in a sane way.
And it doesn't have to be in A to Z. It could be (and IMO, should be) in "special" which such initial-index catalog typically already have.
https://cdn.shopify.com/s/files/1/0022/8193/0805/products/im...
When music downloads were first starting up in the late 90s I was responsible for building a schema to store all the world's music. We had to rip millions of CDs. When I got that Aphex Twin one above I gave up. I think we just called it [Track 2] or [Maths symbols] or something.
The reason I mention this is because you cannot simply look at a list of all edge cases, identify what schema you can use to encapsulate them all, and then build your service around the idea that the schema will never change.
As a very simple (I hope contrived) example: You could decide to use a string delimiter so that you can capture "all the strings" and still use string interpolation. You then look through all the edge cases and see "No one has named anything with U+1F7F6, so we're safe to use that". 100% guarantee that as soon as someone sees that they will name a track or an album in the exact way they need to to break your system.
:-D
Artist: Moby
Title: Simple Love (B-Motion Remix)
The actual artist is B-Motion and any kind of browser/catalog must group this track with other B-Motion music, or in worst case both, because remixes are in entirely different genres than the original.
So "Artist" is not necessarily the artist.
- And for song titles that need careful escaping, QuelI->EX[cez]->{kranz}; and EXEC_SOL=FAGE/. have tripped up more than one CLI music player in my experience.
Features like long album names, empty names, or names with international characters from Unicode, are not hardly edge cases. Edge cases have to have some surprise to them.
Coming from a blank slate, you'd never approach a problem with the assumption that all strings are 32 characters or less, confined to 7 bit ASCII, and are never empty and then paint your face surprised call "edge case!" when they aren't.
J.K. Rowling has written works under other names, e.g. Newt Scamander.
And then you have the case of the (actually recorded this way by the Library of Congress) author "Mark Twain (ghost, through mediumship)"
Sometimes the music app just smooshes all together so the "album" looks like it has 4 of each track
Sure, if you ignore blues&jazz, traditional folk, and all non-western music.
'Twas all in vain, since I wanted a particular remix of `Alright' — which I have on a vinyl promo less that a metre from where I am sitting (I can remember clear as day buying it more than 25 years ago) — which Tidal don't have. So I ended up listening to it on YouTube.
Another example... from 1976 to present day: Johnny Cougar → John Cougar → John Cougar Mellencamp → John Mellencamp.
Close enough for library software to store/recall them all as "John Mellencamp"? Maybe. But perhaps not to fans or historian or artist liking.
(for me, 'artist' is the alias, 'album artist' is 'Richard D James'...)
It was all done when added to the catalog, but presumably the metadata could be updated and a periodic refresh would be a benefit.
∆Mᵢ⁻¹=−α ∑ Dᵢ[η][ ∑ Fjᵢ[η−1]+Fextᵢ [η⁻¹]]
With the ∆M and the sum over Fs, I wondered if it might be something to do with rocket dynamics derived from Newton's second law?
https://i.discogs.com/YC8CLuli-NW3VB5z7eA9mALjnL2Qef4aGyDYH4...
As an example, why would the index of the last term be inverted like that? Maybe it's supposed to mean something phonetically, but I'm not a music person so I can't figure it out.
Even in English-language contexts I've seen variants of the German-style transliteration on occasion. But more importantly, the author is not restricting himself to English, and indeed the musicbrainz link gets to such a high count precisely by including other languages' transliterations. Only three are listed as English, and they're the two you mention (with/without Anglo first name) plus one that omits the middle name.
Were you saying that all other languages should use "the modern accepted spelling in English", or were you just ignoring the fact that speakers of other languages (or owners of foreign-published albums) might use music databases?
This is a good example of what happens when the insensate take power and pander to those who just want to read the "rule book".
No art. No expression. No bother.
Let's just agree on some fucking fiction that gets record label to bother with classical music meta-data in a way that meets the real world. Can we start there, at least?
edit: click the inverted triangle if you are a jack-ass