You should probably first consider seeing if there are critical bits of code that can be rewritten to use MMX/SSE instructions, since your data is in the right cache already, without needing to move it anywhere else.