Is it because the field of CS has advanced further, which means that we can apply modern ideas and algorithms?
Or is it because we can cheat and have a toolchain running on vastly superior hardware, which enables us to use it in ways that would have been too time-consuming to develop back in the day?
It also helps that the toolchains are far superior to what we used to have .. I never would have thought, in my youth, that I'd be writing C code for the Oric-1, but indeed the OSDK has a C compiler that makes the effort of packing code into 48k a lot more fun than it used to be!
But in the end, you don't write software for these old machines unless you have a lot of passion, and that's what counts, I think. There is a lot of passion for the Oric-1 these days, that is for sure .. at least in the small scene that keeps the machines alive.
I'm not sure I agree about toolchains, though. The 6502 is not a very good target for C compilation, pretty much any assembly programmer can code circles around what cc65 produces. So, no offense to you or anyone else that writes C for 8-bit machines, but I don't think any of the new games and programs that really "push the limits" of these machines are written in C. So that leaves modern cross assemblers, which are indeed much more capable than common target/native assemblers of the olden days, but don't exactly afford a quantum leap of productivity either.
Then again, modern emulators are a godsend for debugging.
Hell yeah they are. I think thats also a key factor for why development on the old machines is experiencing a revival.
today, we have plenty of programmers who have been at it for thirty years. They got started on these very machines, and in the meantime they have been doing other things on other computers—but which were still programming—and now, if they decide to go back to the original computers, their instincts more than compensate for the lost familiarity.
In some of my last years as a student (late nineties), I dug up my old ZX81 from my parents' basement, and the effects of all three items were visible: I had access to a cross-assembler that I ran on a SPARC workstation. Techniques that may or may not have been known in the 1980s were now published and available on the Internet for anyone curious about them. And my ability to concentrate and understand the issues I was facing was better than when I was eight.
Yes, theoretically you could stick an ARM processor in a cartridge and run most of the game off that, using the PPU as a glorified framebuffer. But many (most?) retroprogrammers are in it largely for the engineering/optimization challenge (or else, why wouldn't you just write a retro-styled PC game), so they draw the line for "cheating" much more strictly than even you have. e.g. some fan translators refusing to expand the ROM size of a game and butchering the translation to fit (Japanese text is much more dense than English), instead of using a larger ROM to fit a better translation. Or some NES homebrew programmers not using mappers at all and working within the constraints of NROM (32kb program rom and 8kb graphics rom). So I wouldn't accuse any of these homebrew programmers of "cheating," because if they were actually guilty of it, those in "the scene" would have called their bluffs long ago.
But if a retroprogrammer were to use a mapper to achieve a 1 meg ROM, modern hardware would make that a decision with very little negative consequences. A programmer at the time might have to live with a game that cost well over $100.
So cheating in quotes because people did it in their actual cartridges. Cheating without quotes would be something like breaking out of the emulator sandbox and embedding the Quake engine.
The edit-compile-test cycle is near instant and debugging from an emulator is a lot easier than debugging on the real hardware.