Except likely larger and with less durability and battery life, yes?

I had an old Motorola Droid I used to do this with, then switched to my Windows phone without any issues. Battery lasts a few days when you turn everything off, and the music app's interface is clean and easy to use. I just dropped a 128GB card in with all my music and podcasts and I was off and runnig.

You can find several older Android and Windows phone models for around $50 or less on ebay and most of those early phones had replaceable batteries as well. Grab and extra battery and you should be set for a few years.

I did this but be vary of poor head phone connectors. My Nokia 130 started panning the sound randomly after a year or so and ultimately gave up altogether after two.

Usually these oldschool phones do play music and you can manage songs quite ok with files/folders on a micro sd card. What can be annoying though is a terrible UI for playing your music, don't expect things browsing tracks by artist for example...