You need not buy a new phone. Some of the Samsung phones of the 2005-2006 era were my favorite, in terms of speed and menu design. They can be found on Ebay for <$10
I agree. And, if you have a phone that is too "smart", you can always root it to edit what is easily available to you.
I'm sure that YMMV. But there are often problems activating very old phones. E.g. Verizon won't activate phones that don't meet their GPS requirements.