Familiarize yourself with the tools for measuring performance on whatever platform you want to optimize for. Then you can just start messing around and seeing what kinds of things make the number go down.
And this will almost certainly require a deep understanding of the instruction set, and dropping into inline ASM, periodically.