Rockbox[1] might be able to help you out with this - it's supported on a fairly wide range of devices, and includes an alarm-clock feature.
[1] - https://www.rockbox.org/
[1] - https://www.rockbox.org/
The Sandisk Fuze and Fuze+, Clip, Clip+ and Clip Zip all have physical buttons and great sound quality for around $40 [1]. Note that newer iterations like Clip Sport Go use different hardware and are incompatible with Rockbox [2].
[1] https://www.ebay.com/sch/i.html?_nkw=sandisk+clip
[2] https://www.reddit.com/r/rockbox/comments/ap3ze1/trying_to_i...