I'd go this direction as long as it had a real keyboard... Maybe it's time for the blackberry to reemerge.
The other way to go is with an ultra-small laptop (typically 12--13"), a tablet with integrated keyboard, or a (usually DIY) cyberdeck.
Gpd makes smaller machines than that.
This is a 9" (well, 8.8") device (display).
Though it looks as if there starts to be a price premium. I'll grant that the specs are impressive.
A bonus is that it has some features for interacting with modern interfaces, like touch sensitive scrolling by swiping along the keyboard.