Building a digital music collection in 2023
worthe-it.co.za
worthe-it.co.za
I used to think this way, but I had one too many things I acquired, then deleted because I wasn't really listening to it, then years later suddenly wanting to try it again because something just clicked and spending many, many hours having to re-acquire it.
Storage is cheap. I just keep everything and buy a bigger hard-drive.
Most of my headaches these days are around whether I correctly tagged something the first go-round and now having to sync all my backups when I decide I want to reorganize some of my albums.
Let's just say 1.5 TB worth. Mostly 192kbps mp3s and higher, but maybe 10% or more FLAC.
One idea that keeps me up at night is a collection that expires itself. It’s probably like an LRU cache for music, reflecting a life-cycle that’s self-contained.
There’s a passage in Infinite Jest that describes the logical point that a quantity of meat exists that I will eat over the course of my life and it could be stored in a substantially large enough room.
Similarly with our music collections, if you will, there’s only so many tunes left. The excess is what is wasteful.
I don't know if I'd quite put it that way. Maybe infeasible for one application or another (I have no experience with Kodi). But it's not hard to simply have that many files on a hard drive, in a standard Artist/Album directory structure.
Assuming 5MB per .mp3, 1.5TB is about 300K songs. No problem for a standard filesystem these days.
Maybe mirror it with another 1.5TB drive for safety, and you should be good to go for a long time.
For a long time I only collected complete albums. If there was one song I liked I would acquire the entire album and digitize it into my collection. Some of the other songs would go unlistened, and some I even disliked, but I keep them all the same. Over the years many of those other songs have grown on me through their appearances during shuffle-mode sessions, or I've come to appreciate their connection to the other tracks when listening through an album as a whole.
Nowadays a lot of music I listen to is released without anything like the framework of an "album" so I have a harder time using this method, but still I'll often acquire additional tracks (or an entire discography) from an artist I find interesting with no immediate intent of listening to them.
Git is a terrible solution for big binary files that never change.
Put your lossless files on a server as a source-of-truth (with a regular cold backup somewhere else) and install a streaming server, like Navidrome[1], which will allow you to transcode on-the-fly to all of your devices. This is how you build a true "digital music collection": so that you won't regret it 5 years from now, when the site you bought your flac's from closes down/erases your files (as has happened before on Bandcamp when an artist account gets deleted!), leaving you solely with the reencoded mp3/opus files you kept, unable to move to better formats as they progress.
If you're already using them, both Plex and Jellyfin also serve as music streaming servers (Symfonium will also work with these)
There's several other open source Android apps as well. I've been testing out as many apps as I can find and documenting the features of each in this spreadsheet[0]. It's a bit out of date, and there's one or two more apps to add, but you get the gist
[0]: https://docs.google.com/spreadsheets/d/1Lr788M1pM3AFOpjR8CRA...
There are drawbacks to any collection method, physical or digital, and at some point those drawbacks start to wear on me and I start to loathe the collection process. After enough loathing, I get rid of it. Or if transferring it between computers or apartments is more work than I'm willing to put in, I get rid of it.
Sure I've had to buy the same album two or three times on occasion, but not nearly as often as you might suspect (especially now that streaming services are a part of my listening habits). I'm much happier now that I'm not meticulously collecting and worrying about backups and such. I have an ever growing MP3 collection which is mostly stuff from bandcamp (and other artist-friendly digital sellers) and some ripped CDs. But I don't back it up, and I won't transfer it to my next computer (though most of the digital purchases are re-downloadable, at least for now). I stopped completely with physical media after living in a few very small apartments, and since I no longer have a turntable or CD player (or a CD drive, for that matter). So I just do streaming services or local MP3s (not meticulously organized -- I can just search the HD). It's not perfect, but neither were all the other methods I've tried (which include many of the systems people have suggested in the comments here), and this is far less work for effectively the same result.
If anyone's feeling collector-burnout, I highly recommend just getting rid of your collection (throw or give away, don't try to sell the sell-able stuff) and starting again simply.
I started collecting on bandcamp and it was really refreshing. I picked up a few albums I used to own, but maybe only 1-2% of my collection. I like how they show you music that was purchased by people who share some albums in their collection with you.
I tried streaming for a few weeks. It didn’t feel right, like I was ripping off the artist. I’d rather support bands I like by buying their album. I’d have to stream them thousands of times to give the same level of support as an album purchase.
But of course I’m just a hypocrite because I stream television. Then again, I never collected video.
I've digitised my collection and sold off physicals, in no way would i ever delete the digital copies!!
That what I used to think too until a (not backed up) hard drive with my entire collection died. At first I was devastated, but then I came to accept that it wasn't that bad -- for the most part, all that music is still out there somewhere, so if I really wanted to hear it again, there was probably a way. Meanwhile I was still listening to and collecting new music anyway, so my collection grew again. At some point later I upgraded my computer, and I intentionally didn't bother transferring my digital collection. It was fine. It doesn't need to be forever. At least not for me.
Nowadays I tend to buy music on either CD or FLAC (from Bandcamp etc.). Storage keeps getting cheaper, bandwidth is cheap, and my bet was always the price of both will keep decreasing to where it's not worth the quality/transcoding issues of using any lossy format. It's now pretty much at the point where it's possible to stream lossless audio from the home server to whereever I happen to be (with a backup of an MP3 player with a big SD card whenever there isn't bandwidth).
Yes! I only buy music on CD or FLAC. Nothing else is good in the long term.
Sadly, when I first digitized my music collection around 1995, I didn't have that foresight and encoded pretty much everything as MP3s, mostly at 192kbps. And they sound poor enough to irritate me now.
My collection was large then (and is even larger now), and I still haven't finished the project of digging those old CDs out to encode them as FLAC.
I used to use Subsonic on my phone to listen to my music directly, but Apple Music is so nice since it seamlessly merges my main library with its streaming stuff. Nowadays I seem to use Apple Music for like 90% of my "discovery". And I still slowly buy up songs that end up making the cut.
I'd like to transcode to Apple Lossless and have _that_ in the Apple Music cloud instead of AACs but I don't want 500 GB of music on my laptop. Somehow I can forgive 80 GB, but 6x that crosses my line… Perhaps I could use a mounted network share as my iTunes library, but I suspect iTunes won't be happy when part of the backing store constantly goes away and comes back later.
My sweet spot has been Plex + Plexamp. I have a truenas server and created a 'media' nfs share for my Plex instance. I've been a very happy camper with this set up for the past 3 years. I can expand my storage at any time, ZFS is very powerful and gives me some backup options. Plex effectively replaces all my streaming service needs.
What raid z are you running that lets you expand on the fly?
I apologize if this is a stupid question, but I really don't know. I've just been pressing "play" on my streaming service for so long, I forgot all about actually "owning" music.
I buy my music on CDs or through a service like Bandcamp that lets me purchase music directly in a non-DRM lossless format.
I've also had some good luck buying music directly from the websites of the musicians that made it.
> Do I need to purchase a CD, then rip it myself?
That's what I do if I can't find a good encoding for purchase.
> If so, are artists generally still pressing and distributing CDs?
A lot are, yes, but it depends on what sort of music you're into. Another great source, if you like small indie bands, is to go to their concerts and buy the CDs they always have for sale there.
> I recall Amazon Music used to have a "Download mp3" feature, but not sure if they still offer that.
They do, but don't do that. Their MP3s are poor quality (and you don't want MP3s anyway. Look for something lossless like FLAC).
Ideally, yes and yes unless you can find it in a lossless, DRM-free format.
For best accuracy, use something like EAC or dbPowerAmp to rip.
Mainstream stuff, iTunes has the biggest collection, but Qobuz and 7digital more reliably have lossless (a lot of the iTunes long tail is stuff encoded in the mid-00s) and work without a dedicated client app.
Personally, I buy from https://bleep.com/ , I've been using them for well over a decade, I find it excellent for indie and electronic stuff, and still buy random albums from their recommended section if I want to try something new.
All the great tools out there for tagging etc are ALBUM/ID3 oriented. I use Beets and it's really good for when I want the whole album.
But now I'm realizing: there's a lot of songs I have for which I really really don't want the whole album. So I have this massively old collection of random mp3's with weird tags. Anyone come up with tips or ideas how to begin to tackle something like this?
(edit, again, I don't much mean like "what software," more like -- how do you organize this? I keep coming up with decent possibilities, but none perfect and none that I can motivate myself to work on?)
I've done a few passes with my own scripts to get the tags a little more sane, but it'll never be 100%, and I don't spend too much effort on it.
For me, I keep things in Artist/Album structure, mostly. But there's always a "Misc" folder for each artist, for the one-off stuff.
There's also a top-level "Misc" folder for truly random tidbits.
For streaming in my home, I use Logitech Media Server (formerly Squeezebox), and have been happy with that for many years. It's open source, so I can always build my own player from a raspberry pi in the future if I need to. I tentatively hope it will last me the rest of my life (:
Sometimes I just randomly search LMS for some text and see what songs come up, to find random stuff.
Or sometimes I search on the filesystem using normal unix tools (`find`, `grep`, `locate`, etc). Works for me, but I'm not a major music nerd, either.
What I really need is a way to be notified of releases from bands I want to track, and rate songs that I both do and do not own so I know what to ignore or investigate further. Of course this needs to be via something I myself can optionally own/manage, not just through a third party that will enshittify.
I’m not a huge fan of e-mail so I convert all my Bandcamp emails to RSS[1] which I self host with fresh RSS.
I have found that my wishlist on Bandcamp has grown to over 500 albums as I mostly use it as a “to listen to” list. But I find I need a place to rate and comment privately on my wishlist. I recently wrote an app called camp counselor[2] to help me manage my wishlist.
[1] https://blog.line72.net/2021/12/23/converting-bandcamp-email... [2] https://line72.net/software/camp-counselor/
I'd be leery of storing the music files in a git repo directly as MP3 tags and metadata are a wild west and some music players do wacky things like update ID3 tags with play counts, etc. on every listen (which would require committing and pushing music on every listen!).
I settled on syncthing instead. it has given me few troubles.
For example, if you want to protect your music collection from accidental changes and are willing to use it manually from the command line, git-annex will work better. Or when I used it for my music, I had a branch for all my lossless files, and then a separate branch for curated and transcoded files that I used for syncing to devices with less disc space. (Perhaps not the most useful example, but the point is that there are a bunch of git-annex workflows that have no parallel in SyncThing or similar tools.)
that's what plain old backups are for, surely? if I screw something up I can just restore from backup.
and anyway this is just not something that has ever caused me a problem in the first place. how often do you even do something that has a chance of fucking up your music collection in particular?
PS- beets (beets.io) is an incredible tool for ingesting music in a standard way. My typical workflow is to dump albums into an inbox and ingest them further using beets. beets maintains a library db which I could see managing with Git, but idk why you’d track the raw files when you could use something simple like rsync to backup to a coldstorage location (either NAS or cloud).
I have a 2 step process for this. The first is an Automator script that lets me transfer files from my Mac to my Debian server very easily—right click a file (or group of selected files) in Finder and my script shows up under 'Quick Actions' "Copy to <my-server>::tmp". [1]
The second step is an "import" script on my server that unpacks the zip, grabs the meta data from the songs (including the cover art which Bandcamp provides), and shoves it all into the correct place. There's not much to share here because it pretty bespoke (and also kind of ugly internally since I wrote the first version almost 20 years ago--though the years of debugging have made most of the end-user experience pretty smooth). It sounds a lot like what beets does.
[1] https://porkrind.org/missives/automator-is-pretty-sweet/
Semi-relatedly I really like the CD collection portion of your website! Makes me want to implement something similar. With that said, I'm a little concerned that your Aphex Twin selection doesn't include Drukqs or SAW85-92, and that your Paul Oakenfold selection doesn't include Bunkka....
> I'm a little concerned that your Aphex Twin selection doesn't include Drukqs or SAW85-92
:-). My Aphex Twin phase was pretty short (I bought the 2 albums on 2003-01-03 and 2003-07-05, both from Fry's, according to my metadata :-)), it ended up with me deciding it was a little experimental for me so I never pushed too much further into the discography. It was kinda hard back then—it was committing to buy a CD usually sight unseen or listen to 30 second preview on iTunes (which is hit or miss depending on which 30 seconds of the song you hear). I'll check them out now with our modern streaming services :-).
> your Paul Oakenfold selection doesn't include Bunkka....
That's one of the few albums where I bought just a couple tracks on iTunes. I didn't like the idea of a non-complete album in the collection so I stuck those all into 'Various Artists/iTunes Store' [2] (there's also a similar "album" from another somewhat sketchy store that doesn't exist anymore [3]).
[1] https://metacpan.org/pod/Apache::MP3
[2] https://music.porkrind.org/Various%20Artists/iTunes%20Store/
[3] https://music.porkrind.org/Various%20Artists/All%20Of%20MP3%...
Git is storing at least two copies of files: one in your working tree and one as a zlib compressed blob in the object store.
It may also store more copies if, for example, you updated embedded metadata and then have to recommit the file.
Don't transcode. Please.
# find ./ -name "*.mp3" -exec ./to-opus.sh "{}" \;
Even more so as the script doesn't even add a metadata tag to document that it is transcoded from mp3, as other scripts do.
As for "smaller files" fuck that shit. For the longest time, I went with 320kbps mp3s. I was limited in the space I could dedicate to music (a few hundred gigabytes). But now? Storage is only cheaper. Get FLAC for everything. And the cloud? I think that's called "someone else's computer" or so I've heard. Stream/sync it from home.
Tempted to do the same for my in-home setup to lock the sample rate at 48KHz - my tired old home server struggles to down-sample the higher resolution FLACs that I have before throwing it to SnapCast.
For listening I figure I’ve spent so much $$ on my setup I’d be dumb not to ensure the source is not perfect too - for free.
I only wish I could do the same with video.
Before, I've organised things like `~/Music/Artist Name/Some Album/A Song.mp3`, and just used the media player of the day. But, that doesn't work great to find stuff by a particular musician when they tend to get together with different sets of other musicians to record one or two songs in any particular arrangement.
Even uncompressed PCM is cheap enough to store that I see no good reason to convert to a lesser formt, or even convert to a more efficient one.'
I'll go for FLAC if available, or WAV or highest bitrate whatever lossy crap alternative.
Even with large file support, GIT is not a nice way of storing music, I don't need any of the that git offer here..
My directory structure is "put it somehwere under /data/audio/music" if I have multiple albums from an artist, put it under /data/audio/music/artistname
I let my scraper build an inventory from the metadata, when I'm on my own LAN, I run rhythmbox against it, when I'm out, I use a bespoke player to navigate it from fuzzy searching and building play-queues. There's a web-interface so I can play from the browser when at work, and an android app that I can use when I'm on the phone.
Deleting things? I do that only when I've acquired a better quality release, or discover dupes. The only "cloud" principle I use is that I view my music collection as a cloud, it's something I search and explore, not a neat overview.. There's no way to have a neat overview of the infinite.
Unfortunately it's become harder and harder to find high-quality music releases that are readily available for download. More often than not, I have to resort to sites that will rip audio from Spotify/YouTube/SoundCloud, because that's the only place I can find the given piece, and typically that'll come as 128kbps which can be pretty easily noticed. I hope more people push back on the streaming model and encourage artists to sell their music direct. I'll gladly pay an artist for their work but I'd rather not have a middleman (Spotify) involved.
---
Interesting to see OP use git as a storage mechanism, I can see it may have some version control benefits, but what about actually playing the music? Do you just click through the file explorer to find the song you want? What about playlists, and other means of categorization?
The font is fuzzy and it isn't a strong black font colour.
Though about the subject of the post I too am still kind of a retro-grouch regarding the music I listen to. Everything is stored in my computer and I listen to it with ncmpc - even did a little dumb patch to change that horrible progress bar with a straight thin line, but I like it more than ncmpcpp that feels kind of bloated.
I got a bluetooth speaker so I can "stream" to it when don't feel like playing it from my computer for whatever reason, and with KDEConnect can control it with my phone. I'm using mpdbus[0] to bridge mpd to mpris and kid3 to tag songs.
No longer carry around music in my phone but when I did everything was in a micro SD card. I got an USB adaptor for it just in case, so I guess that makes it for the "streaming" capability.
Ref. patches, I use a cheap Amazon RF remote to control MPD - essentially a bit of code that receives button clicks and controls volume. forward/back etc. Low tech FTW :)
With regards to Dropbox, there's a simple free tool we've put up here: https://asti.ga/dropbox-music-player . It's useful in the sense you can playlist, loop, shuffle etc although you don't have the full library browsing experience Astiga offers.
Soulseek - slsk.org
Information wants to be free. The artists get almost nothing from sales anyway and would like you to listen and enjoy regardless.
What you wrote may apply to huge artists like Beyonce or Taylor Swift (although even then they make a non-negligible amount of money off of sales since they do it in such huge volume). And it's true that artists make very little from _streaming_. But sales can be a large portion of income for smaller-than-Beyonce artists, especially if you buy physical media direct from them or digital media from places like Bandcamp.
Especially if you buy digital from places like Bandcamp, artists can actually get a significant portion of what you paid. I think when you break it down, buying digital from Bandcamp nets the artists just as much (if not a little more) than buying a CD off their merch table at a show.
Finally I assure you, most artists would prefer to get paid AND have you to listen and enjoy at the same time, instead of not getting paid and having you listen and enjoy.
I wish soulseek-qt just let you hide all locked content from showing in the UI, so I'm not worrying about what I'm missing out on by not interacting with rude bitter assholes who want payment or trade.
Anyways, your link is a 404, the correct domain name is https://slsknet.org
A totally different story is: "I pirate because I can and because I want", but do not try to pass pirating as some sort of utopian "information wants to be free" BS