I've actually been working on something of this sort with a battery-powered Raspberry PI running Debian and a Pervasive 7-inch e-Paper display. I whipped up a pretty decent prototype that had a terminal (so, Emacs, w3m, newsbeuter, etc.), PDF reader, and rough support for keyboard-driven X apps, but then my little girl got to the wiring with scissors, and my momentum died. (It also had a small character LED display to show rapid feedback while typing.)
The last few days I've been working on a new version of the software, and I think I'll wire it back up. If I get it clean enough for others to use, I'll write it up.