iPods became big around 2003. Back then, 128 MB of flash would cost you $40[1]. This was why HDD-backed mp3 players were so revolutionary for the time--flash wasn't viable, yet. By the time flash got cheap enough, portable mp3 players were so established that the time for physical media for audio had mostly passed.
1: https://web.archive.org/web/20030520151144/http://www.newegg...