For straight up modding: definitely the Xbox. The 007 and Mechwarrior bugs blew everything wide open, and the fact that it was just a PC with real (upgradeable!) storage spawned projects like XBMC, now known as Kodi: https://en.wikipedia.org/wiki/Kodi_(software)
And also piracy was rampant, but not the Swapmagic or Modchip kind. You could just upgrade the drive, _backup_ your games on there, and play 'em all of the drive.
The Wii and 3DS are also suuuuper open and hackable though. The homebrew scenes on both are incredibly impressive, not to mention the whole ecosystem of full blown launchers and shells and stuff. (Which, now that I think about it, was also a big deal on Xbox.)
For a lot of games, it totally didn't matter (shoutout Ikaruga, 38 MBs! https://www.thedreamcastjunkyard.co.uk/2023/03/the-worlds-sm...)
But for games that took advantage of the extra 300 MBs, pirates had to use all these tricks to get the game down to a CD-R size. They'd compress assets, compress or sometimes rip out the FMVs...I think they might have even split some games across multiple CDs.
That's why DRM cracks me up, the pirates will always figure a way around it one way or the other. (Especially in today's day and age where the live service model is so effective. I'd weep for the AAA single-player game, but I can't remember the last one I played and enjoyed. They've been dead for a long time. Long live the indie single-player game.)