My music collection is a giant pile of files (~19k tracks, according to iTunes) that I have been curating for a similar length of time, but I let iTunes do all the grind of organizing them on disc for me. My purchasing and listening is pretty much entirely on an album basis.
iTunes uploads it to Apple's cloud, since I'm paying for iTunes Match; my iPhone and iPads pull it down. It is very easy to search for a single album on the iOS players; they also make it very easy to find your most recent acquisitions, with an automatically-populated "Recently Added" playlist.
~/Music/iTunes/iTunes Media/Music looks almost exactly like the directory structure you're probably meticulously maintaining by hand (lots of Artist Name/Album Name/01 Track Name.m4a), but adding new stuff to it is a simple matter of dropping a directory full of properly-tagged files onto iTunes. Then I delete the original files after iTunes has copied it into its directory.
It breaks down if 90% of your collection is a bunch of badly-tagged files you downloaded off of KazAa, but if your collection is a mix of stuff you ripped from CDs back in the nineties and made sure were tagged properly, and stuff you've bought that the musician/store tagged properly, it works pretty much seamlessly. Except for this one weird glitch where sometimes iTunes on my Mac decides that I have both the copy of a track I bought off of the Apple store, and one in the cloud, and thus plays every song off an album twice. I've kinda quit buying stuff from Apple because of this; I'll go to Bandcamp first.
What is the attraction of meticulously maintaining a directory structure that a computer can very, very easily maintain based on the metadata stored in the files? Why are you so married to explicitly browsing a duplicate of this filesystem on other player devices?