I love custom handheld computer projects like this.
Few years ago I wanted to build one as a hobby/toy project with parts that are more or less easily available. So I did [0]. Instead of using a pre-made keyboard I used simple push buttons and instead of specialized keyboard controller I used an Atmega328P. Most of the components are through-hole and easy to solder. Anyway, the couple of the handhelds I built are sitting in a drawer at home, but it was fun building it nevertheless.