I buy a lot of music on Bandcamp and at shows. Other stuff (older stuff) I buy at thrift and music stores (there are many where I live). I also started out with a large library due to CDs that were handed down to me.

Soulseek is good if you wish to sail the seas, Bandcamp for indie musicians getting paid fairly.