Beets – Command-line music manager and auto-tagger
github.com
github.com
Tip: If you are about to import all of your music you and you have enough space you can save some of the tedium by doing a quiet import initially:
1. Optional: `$ cp -r ~/Music ~/ImportMusic ; mv ~/Music ~/Music.orig`
2. enable whatever plugins you want (atleast: chroma fromfilename fetchart)
3. Set up config
directory: ~/Music
import:
move: yes
quiet_fallback: skip
log: ~/.beets/imported-beets.log
4. `$ beet import -q ~/ImportMusic`
This will take a little while depending on how big your collection is. When that is finished the only things left in ImportMusic will be the things beets need some help identifying. This part requires time and a little input from you so plan on being near your $TERM 5. `$ beet import ~/ImportMusic`
Keeping the backup copy of Music is probably not needed but I like to play it safe.tldr: No matter how you want to import do the quiet+skip import initially.
ADDENDUM: A similar program for pdf/epub/mobi --boox-- would be awesome. If Calibre worked half as well and was developed by someone half as nice as sampsyo the UI would almost be tolerable.
I've uploaded our media management scripts to GitHub here[3]. Note that there is some minor rude language in a few scripts ;)
In case of failure of Google I still have a decent amount of music on my phone and I do have the stuff on my phone. In the (much more) likely case of drive failure I can get my music again (I could even circumvent the download limit through a Google->phone->PC thing but that would be a pain).
I think it's important not to overestimate your personal capacity to not lose things compared to companies like Google (especially considering that I am not aware of many companies that fell off the face of the earth in less than 24 hours for you to lose all your data)
edit: google music player on android is pretty annoying though (just slightly better than the stock player from old android). The fact that you need a data connection to play a random mix of your music startles me.
>I don't want to circumvent anything in order to have access to my data.
If you're an audiophile/actually doing things with the raw files, then this is not for you, but for most people (read: 99% of people) the diff between 320kbps mp3 and flacs is not there.
> Why does it startle you? When you don't pay for something you are the product. I would be more surprised if google gave you all this great music and did not expect anything in return.
Sure, I don't care about sending my music usage stats(hell I already do it publicly with last.fm) but at least have an option when my data connection is down.
It does take a bit of effort to manage my own music collection, but I feel like I am well protected against data loss by having it synced between a NAS, external server, and Crash Plan. 90% of my listening is done via the external server running Subsonic, using both the web interface and the excellent mobile app iSub. I also try to always pick up a vinyl at live shows since they are big and nice to look at (and occasionally listen to), and usually come with a free mp3 download.
There are no limits on how many times I can access my music files. I can access them any way that I please, from pretty much any device, forever. I couldn't have it any other way with my music collection.
This is a halfway solution for people who are not necessarily good at backups. And I get to listen to the music on the web interface or on my phone (which is basically all I care about). Maybe I'm an awful person, but I greatly prefer this to the USB/SD card shenanigans of yore.
Been using Google Music ever since.
I'd be nervous buying DRM'd content from a supplier that may not be around in the future. But renting I could care less - unless I paid for years upfront - but you generally don't do that.
Sure, they butchered it up in version 2.0 and above, but I've always felt version 1.4 was probably the best library manager I've used.
Clementine vs Amarok
# apt-get --no-install-recommends install clementine
...
The following NEW packages will be installed:
clementine libechonest2.1 libftgl2 liblastfm1 libprojectm2 libprotobuf8
libqjson0 libqt4-opengl libqxt-core0 libqxt-gui0 projectm-data
ttf-dejavu-core
0 upgraded, 12 newly installed, 0 to remove and 19 not upgraded.
Need to get 6,715 kB of archives.
After this operation, 28.1 MB of additional disk space will be used.
# apt-get --no-install-recommends install amarok
...
The following NEW packages will be installed:
amarok amarok-common amarok-utils kate-data katepart kde-runtime
kde-runtime-data kdelibs-bin kdelibs5-data kdelibs5-plugins kdoctools
libattica0.4 libdbusmenu-qt2 libdlrestrictions1 libepub0 libfam0 libiodbc2
libkactivities6 libkatepartinterfaces4 libkcmutils4 libkde3support4
libkdeclarative5 libkdecore5 libkdesu5 libkdeui5 libkdewebkit5 libkdnssd4
libkemoticons4 libkfile4 libkhtml5 libkidletime4 libkio5 libkjsapi4
libkjsembed4 libkmediaplayer4 libknewstuff3-4 libknotifyconfig4 libkntlm4
libkparts4 libkpty4 libkrosscore4 libktexteditor4 libkxmlrpcclient4
liblastfm1 libloudmouth1-0 libnepomuk4 libnepomukcore4 libnepomukquery4a
libnepomukutils4 libnl-route-3-200 libntrack-qt4-1 libntrack0 libphonon4
libplasma3 libpolkit-qt-1-1 libqca2 libqjson0 libqt4-opengl
libqt4-qt3support libqtscript4-core libqtscript4-gui libqtscript4-network
libqtscript4-sql libqtscript4-uitools libqtscript4-xml libsolid4 libsoprano4
libssh-4 libstreamanalyzer0 libstreams0 libtag-extras1 libthreadweaver4
libzip2 nepomuk-core-data nepomuk-core-runtime ntrack-module-libnl-0
oxygen-icon-theme phonon phonon-backend-vlc plasma-scriptengine-javascript
shared-desktop-ontologies soprano-daemon
0 upgraded, 82 newly installed, 0 to remove and 19 not upgraded.
Need to get 75.9 MB of archives.
After this operation, 202 MB of additional disk space will be used.
Gnumeric vs Rkward # apt-get --no-install-recommends install gnumeric gnumeric-plugins-extra
...
0 upgraded, 8 newly installed, 0 to remove and 19 not upgraded.
Need to get 456 kB/7,289 kB of archives.
After this operation, 34.0 MB of additional disk space will be used.
# apt-get --no-install-recommends install rkward
...
0 upgraded, 70 newly installed, 0 to remove and 19 not upgraded.
Need to get 61.1 MB of archives.
After this operation, 143 MB of additional disk space will be used.Sadly I was far from that in my most meticulous days where I wouldn't even delete tracks I hated because that would violate the sanctity of the discography. And I didn't want MusicBrainz to doubt my thoroughness.
That's a bit of a holy grail. Because you need the shuffle to track your mood. I found that it really subdues my listening, in that I don't always want some punk music interupting a mellow interlude. So I've gone back to listening to albums, or crafted playlists.
There are album fillers, and there are tunes that are just bad, and they are better gone. Though granted, the holes don't feel quite right.
I'm trying to understand your MusicBrainz remark, are albums with missing tracks unidentifiable?
The holy grail is most definitely a library (or app) where you click "shuffle all" and enjoy every song it plays. Songza, genius shuffle, 8tracks etc. all try to tackle this, but I still think we're far from that point.
If there are a few obscure mp3s you can't live without you can import them into your Spotify client. They'll get auto synced to any mobile devices you have connected and the client installed on too (although not cloud synced to other computers - you'd have to copy them manually).
Spotify does a lot of local caching, so once you've listened to a track once it'll likely be cached so it doesn't hammer your collection too badly. Pretty sure I've continued to get good gaming pings while my gf listens to Spotify in the next room :)
EDIT: I did not know about the P2P stuff. Huh! Interesting ...
The content is generally pretty good but they depend on the artists/labels getting it into aggregators or making it available. Bootlegs won't be there because that would probably piss the bands off.
https://archive.org/details/etree
Short list of musicians granting permission to have bootlegs hosted on archive.org/etree:
311 Blues Traveler
Camper Van Beethoven Cowboy Junkies
Cracker Dark Star Orchestra
Death Cab for Cutie Derek Trucks Band
Disco Biscuits Donna the Buffalo
Drive-By Truckers Ekoostik Hookah
Elliott Smith Furthur
Grace Potter and the Nocturnals Grateful Dead
Guster Hank Williams III
Hot Buttered Rum Jack Johnson
John Butler Trio John Mayer
Keller Williams Little Feat
Live Music Archive Local H
Lotus Matisyahu
Matt Nathanson Max Creek
Michael Franti and Spearhead moe.
My Morning Jacket Of A Revolution
Perpetual Groove Phil Lesh and Friends
Radiators Railroad Earth
Ratdog Ryan Adams
Smashing Pumpkins Sound Tribe Sector 9
String Cheese Incident Tea Leaf Green
The Breakfast Umphreys McGee
Warren Zevon Ween
Yonder Mountain String Band
My personal favorite is Danny Schmidt, a singer songwriter rom TX/VA: https://archive.org/details/dannyschmidt2007-12-13.sbd.flacThe 4,700+ other artists: https://archive.org/browse.php?collection=etree&field=%2Fmet...
24bit flacs: https://archive.org/search.php?query=%28format%3A%2824Bit%20...
Many bands allow audience taping but I'm not aware of m?any that allow for commercial distribution of those recordings. If Spotify attempted to distribute those recordings against the bands' policies then it would probably piss off the bands.
That Danny Schmidt is pretty great stuff. You're right about Spotify's lack of content when it comes to stuff like this-- they have all his studio albums but are kind of weak when it comes to live stuff, they have one album. Archive.org definitely fills in this gap (I use it all the time for Dead shows).
$ beet etree --get dannyschmidt2007-12-13.sbd.flac
etree: requesting ogg format
etree: fingerprints verified
etree: Success: Downloaded Album: 2007-12-13 - The Cactus Cafe
$
Archive.org has a somewhat documented API that describes all of the "resources." A lot of the shows even come with cover art for those that are into that.JSON for recording: https://archive.org/metadata/dannyschmidt2007-12-13.sbd.flac
Otherwise, where ever I am I have high-speed Internet and I don't bother with my MP3s anymore. I do a lot of Pandora. To play music on the big stereo I just use Google Music and the Nexus 7, which connects to the receiver by Bluetooth.
I used to curate the music, organize the files, connect to machines via ssh to play them back on different speakers, etc etc. Cheap tablets, cheap phones, Internet everywhere, and music services have killed all that for me. I used to have a Linux PC hooked up to the stereo and I would ssh to it to control playback. Those were the days. Now the Nexus 7 is a far superior device in every way.
I kind of hoping spotify does something similar to beats where $15 gets you accounts for all family members. I'm not sure how that works but it seems to be AT&T only so maybe it's subsidized?
If it's more than a couple of decades old, there have likely been a couple of terrible remasters in the mean time - probably easier to get your hands on too.
Rewrite Plugin: The rewrite plugin lets you easily substitute values in your templates and path formats. Specifically, it is intended to let you canonicalize names such as artists: for example, perhaps you want albums from The Jimi Hendrix Experience to be sorted into the same folder as solo Hendrix albums.[^1]
Inline Plugin: The inline plugin lets you use Python to customize your path formats. Using it, you can define template fields in your beets configuration file and refer to them from your template strings in the paths: section of the configuration file.[^2]
[^1]: https://beets.readthedocs.org/en/v1.3.3/plugins/rewrite.html
[^2]: https://beets.readthedocs.org/en/v1.3.3/plugins/inline.html
This makes it useful for playback and finding information without first having to trust it.
Feel free to crib off of my config file: https://github.com/fortes/dotfiles/blob/master/.config/beets...
edit: available only to members, unfortunately
More like literally impossible to download anything :(
I tried a test on a single artist tonight (2pac) but ran into a bit of a roadblock. There's no 'merge' function. So if you have an incomplete album in the library and you import a directory that contains the rest of the tracks, your choices are to delete the incomplete one or to put the new tracks into their own album. This might sound like an edge case but if you have an album that's currently in 2 directories (cd1 and cd2) then you're fairly well screwed.
I am not 100% sure what you mean about merge. You can just `cp LATEADDITIONS Music/artist/ablum` and then "reimport the album" with `beet import Music/Artist/Album`.
As far as cd1/cd2 goes you are only "fairly well screwed" if you can not type `$ mv cd1/* cd2 ; beet import cd2`. I think there is another way to do this but I am not a beets guru. I seem to recall there is a discussion of this in one of the issues on github. But lets face it `mv CD1/* CD2/ ; beet import CD2` is not that difficult.
If you have a big directory full of unsorted albums there is also this new gem:
The importer has a new interactive option (*G* for "Group albums"),
command-line flag (``--group-albums``), and config option
(:ref:`group_albums`) that lets you split apart albums that are mixed
together in a single directory. Thanks to geigerzaehler.I used "fairly well screwed" in the context of trying to automate the cleanup of 60,000+ songs. One album isn't an issue - 10,000 or so where you need to keep dipping in and out of the tool to manually move stuff around isn't really ideal.
Just to confirm, since you seem to have some experience - you can reimport an album to clean it up again (say after adding additional files) by calling import on a path in your library? So it's ok to mess with the files in the library and then fix the db later by importing? From what I saw in the code I didn't see anything that would handle removing / moving files, do you know if it handles those cases?
Again, my language was probably a little strong, and mostly only applies to the more extreme situation I'm in.
ps your other comment about doing the import in 2 steps seems like a bit of a life saving hint. Will definitely go that way when / if I commit to this.
What follows is a slightly rambly not well edited description of my importing. I apologize for the length/grammar.
I think I was in a similar situation as you. I started with 5 or 6 big collections of music that had diverged over the years; flac/shn bootlegs, old itunes installations, xmms libraries, etc and a couple of small collections from netbooks:
/music/itunes
/music/olditunes
/music/olditunes-ibookg3
/music/current
/music/blah
...
Before touching beets I ran rdfind over the itunes directories and then over the non itunes directories. rdfind leaves the copy in the directory listed earliest so I listed the directories in order of most recently used. This essentially wiped out the oldest directory from the itunes and non-itunes sets. I then ran rdfind against all of the directories (once again in order of most recently used). The amount of space I saved was insane.Once I had manually cleaned up the low hanging duplicate fruit I did the two step import. I got hit with the CD1/CD2 quirk the first time I did the two step import. Because I had the backup directory I just blew away the beet library and started over. This time before I ran the quiet import I tried to do a best effort of consolidating the cd1/2 albums. `tree` came in handy for finding the problem albums:
$ tree -i -f -d --prune /music |grep -i "cd1\|disc1\|cd\ 1\|disc\ 1""
I did the `mv cd1/* cd2/` by hand. It did not take as long as I thought it would. I gave it my best effort and reran the quiet import. This solved a ton of the problems. Before doing the interactive import I sorted /music with $ du --max-depth=3 -h /music |sort -h
Because I used the move import instead of copying it was easy to see if there were any recurring problems. The only things that are left are the things beets could not ID by itself. My biggest import hiccup was concerts from archive.org's etree archive. I still have to move them in. I am still deciding how I want to handle the one hit wonder songs. Right now my config has the following for paths: paths:
default: $albumartist/$album%aunique{}/$track $title
singleton: 0xSingles/$genre/$artist/$title
comp: 0xCompilations/$genre/$album%aunique{}/$track $title
The singleton and compilation defaults are: singleton: Non-Album/$artist/$title
comp: Compilations/$album%aunique{}/$track $title
This was going to mean my non-album and compilation album directories had hundreds of directories.Feel free to followup with any more questions.
If you have an album that's split across several directories under a common
top directory, use the --flat option. This takes all the music files under the
directory (recursively) and treats them as a single large album instead of as
one album per directory. This can help with your more stubborn multi-disc albums.
That might be easier than doing all the mv cd1 cd2. I have never used it. It will increase the interactivity of the import.I'd planned on doing de-duplication first. In my case I have legacy copies of libraries - but also a lot of duplication in within the main itunes library (due to exploding itunes dbs some time ago - that was the point I switched to spotify so my ocd didn't kill me). Should be easy enough to script something to handle that scenario. I did something similar a while ago to sort out my iphoto lib(s).
I saw the flat thing (I noticed it in the code). I don't think it would work in the particular case I ran into as the two discs are at the same level. As you say though, if I do a bit of grepping I could probably catch and adjust many of those before I start.
I am using the OS X 10.9 filesystem tags as playlist markers for my music - dirs or individual songs. I can just run my script and it will create m3u playlists off of all the tags I have applied.
I really wish these tags worked in linux. I'm obviously on a mac, and it hurts my heart I can't use it 'where I come from'. Cause I find it ridiculously simple/useful.
find somealbum-with-cds -type f -name "*.flac" -print0 | xargs -0 metaflac --add-replay-gain
Then import as normal and it will have replaygain included (and store the replaygain in the library).The key difference that Beets brings is to unify the concept of a tagger with the concept of a music library, which made a lot of sense in my mind.
In other words, Beets maintains a persistent database of your music collection, which can be kept in sync with Musicbrainz. You can query the database from the command line to get a list of files, and then pipe that list into a music player of choice, or just query the database from a script or web-app and build your own lightweight music player.
It would be nice to have a centralised repository on the website where people can share plugins (like Sublime's Package Control, which is 3rd party). Programmable is great, but it would be a shame if everyone had their own half-arsed solutions to common problems.
[0] http://beets.readthedocs.org/en/v1.3.3/plugins/index.html
[1] https://github.com/sampsyo/beets/blob/v1.3.3/docs/plugins/in...
Anyone else have experience with both, and would like to share their experiences?
I spent days trying to organise my collection of music (and still haven't suceeded). It's in many formats. In the past I've stripped metadata and just relied on file names, and file structure. I've later re-added it, due to helping playback on some devices. Tagging generally annoys me, and doesn't bring me much. The thing I'd quite like is sleeve notes, and good track info, who wrote the song, which artists played on it etc.
Anyway I have tried a mix of quodlibet/exfalso (with plugins), foobar2k(with wine), mp3tag, and shell scripts. Then I discovered beets, and thought wow that's great. Only to later forget about the application and return to the laborious task of sorting out my files. I rediscovered beets and tested it on a directory of music and was pretty impressed with the ease of it all and the results. Far easier than attempting it by hand.
Having said that the bit that takes me the longest is gathering up and organising orphans. If they are albums, I have to get them into order, then try and locate the tags. Or pepper the files with enough clues, for a tagger. It's all a bit of a yawn fest.
Which reminds me, I bothered to apply replay gain to most of my albums with foobar2k but that sometimes results with very quiet playback on some albums on one of my players at least. I then curse profusely, and just want to rip all of the metadata back out, or even just throw the lot away...
"To install without pip, download beets from its PyPI page and run python setup.py install in the directory therein." - https://pypi.python.org/pypi/beets#downloads
"You can install beets by typing pip install beets. Then check out the Getting Started guide."
But it has bitrotted pretty badly.
You could file asking to package the latest upstream and casually mention the discrepancy:) I have just been building beets with uupdate lately. Debian's package is five months behind upstream.
[1] https://www.debian.org/doc/manuals/developers-reference/best...
It says you need an html5 browser to play your music. Is there a plugin that can just play the music directly, so I can play via ssh and not have to kick off vnc?
You only need a browser. On the machine on which you thought you were going to run the VNC client, you open the browser and go to beets’s web UI. Then you pick some music to play, and beets streams it to the browser, which plays it on that machine.
If you actually want the Pi itself to be playing music, then you probably want not the beets web plugin but the BPD plugin, which emulates an MPD server. You can use any MPD client you want to control it – CLI on the Pi, one of the GUI apps over the network, whatever.
No matter what, VNC doesn’t enter the picture.
Why would there be an issue here?
A long time ago I was working on something similar: http://daurnimator.github.io/lomp2/
Got pushed to the side due to study; then work :(