Oh, even 32-bit machines (the pre-ARM Palm PDAs in the 1990s which used the 32-bit Dragonball processor) used standard alkaline batteries (3 AAAs)

Maybe not 8-bit, but my HP 200LX runs months on 2 AAs and is only moderately bulkier (thicker) than a modern smartphone. With a CF card you can have hundreds of excellent DOS programs/games on it.