The PSP has tons of emulators/homebrew + native PS1 support, and you can use Android or PC to emulate the DS/GBA pretty well (same instruction set or JIT-ted emulators). Good thing about Nintendo systems is ROM size, so you can have tons of games and not get bored.

Plus, if you have an old android phone with buttons or close volume keys, you can use it as a complete portable GB/A or DS. I remember to have played the heck out of SFA3/Pokemon in an old phone on long trips.

So I would stick to portable handleds and check out ports or exclusives. If you have a modded home console its more of the same but with better graphics and power or more space for ROMS.