If I am not mistaken, what jimmaswell is describing is the same mod that made my Gameboy Advance my most favorite of systems... Just google 'ds homebrew' and you will discover a wealth of information... For gbas and ds(es???), you could/can purchase a flash cart. There is a surprising amount of homebrew and software hacks available out there that this flashable cart brings to the table - not the least of which is the ability to load up emulators for everything 'equal to or less than' the gba/ds(in general regards to cpu/ram requirements) to play 'backups' of the games (which were also flashed onto the cart along side of the emulators). My former GBA (paired with 2 flash carts) was likely the third most used gaming system of my life to date(2600, NES, GBA). All the games you could want (or fit on 512MB (A LOT when the games are a few hundred k (nes) to a few mb (gba)) - 2600, NES, GBA, GEN and SNES - with few/no issues.

All of this said, the PSP (1001?) was/is an awesome emulator... Capable of playing everything up to PSP backups, you could play all of the above, with PS1, NeoGeo/Capcom, and PSP support... Definitely a very nice gaming system with a lot of power and capability... In the end, it was the GBAs battery life, wonderfully thought out screen and compact size that always made me come back to the gba. The PSP got a lot of play, but being able to stomp turtles and collect rubies while sitting on a beach without shade just made the gba the general purpose go-to for this casual gamer (as the PSP could serve as little more than a mirror and hand warmer in this setting).

While generally frowned on by gaming manufacturers for obvious and founded piracy concerns, there are some awesome hybrid systems that can be made out of existing on-market systems. While I am sure that there are many whose ultimate goal is piracy, there are a lot of people who enjoy modded systems for their flexibility and enhanced capabilities.

There are flash carts for the DS like the DsTwo. It's a custom-made DS cartridge with a slot for a microSD card, which is where you put the OS of the flash cart along with any roms/software to run with it. Getting one for the old DS/DSi line is a safe bet because their firmware isn't updated anymore. Firmware updates are liable to brick flash carts.