The golden age of this stuff was when I was writing stuff for the Atari ST and Amiga. Fixed hardware, manuals that came with assembly instructions and timings, etc.
My greatest moment was when rewriting the Atari ST text output function. For those that don't remember, the ST had an interleaved bitmap [1] and no hardware blitter support, so everything was done with the CPU. Therefore writing text on the screen was hard. I wrote a program that speeded up the C code from the OS about 3x, then Darek Mihocka with Quick ST [2] beat my speedup. I was absolutely aggrieved about this, but spent weeks working out how he did it without any luck.
In the end the solution came to me genuinely when I was dreaming - the Motorola 68000 move.p instruction which worked quite well with the interlaced bitmap. It gave me a nearly 2x speedup and the rest of the optimisations I had meant I beat his code.
I do miss those days of a level playing field where the outcome was the result of your skills, rather than quibbling over the middleware, OS, graphics driver version used, etc etc.
[1] http://codetapper.com/amiga/maptapper/documentation/gfx/gfx-...