The DuraXE does look like a good phone though it has imho non-dumbphone features like wifi. I guess I better check its network coverage and will look into it. According to PC Mag it runs Android 5.1 under the covers, which is a little bit scary to me.
A Baofeng ham walkie talkie is about as rugged as a typical "rugged" phone and costs around $25 so I don't see why there's no phone in roughly the same format. Until the 2g shutdown you could buy 2g phone modules for a few USD, that had some analog audio wires and some control lines and a power wire, so you could make your own phone around it pretty simply. Sparkfun had a product that was one of those built into a vintage rotary phone. I don't think they have those modules for 4g or 5g but I should look again, I guess.
Added: aha re 4g modules, this is $60+ but looks interesting, and makerfabs has some other neat stuff: