Just be glad you don't work on high-frequency trading systems if "tenths of a second" seems like a frustratingly small gain. Our office erupts into cheers when we shave a microsecond or even 500 nanoseconds off our loop times. :)
I assume any kind of OS or HLL is out of the question.
It's a competitive sport, so given the current resources and technology, how much performance can you eke out is really what I should have been thinking about. In an adversarial environment like you describe, I'm sure every bit of time you gain is a huge plus. So yes, our speed doubles every 2 years, but so do our competitors'.