The greenest instruction is the one you don't run in the first place. Remember that the next time your reason to be lazy about perf optimization is "just throw more hardware at it."
Or is the number of people in fixed supply with close to fixed CO2 output and more optimized source code just means less source code, or more people optimizing source code and less people working in the hardware supply chain, or something else along those lines.
Edit: To be clear, this is a real question. I've been mentally debating for quite awhile whether or not I should allocate workers CO2 output to companies or not.
I also think that workers should be responsible for the CO2. Because otherwise, who is responsible?
There is of course always some nominal minimum power consumption for a server to be on. But every instruction requires some additional amount of electron flow to execute.
Look at it another way: if a machine is already in its highest power state, with all cores running, but only executing one thread per core, running another thread on each core may cost almost nothing in terms of energy but could boost instructions retired by a lot.
The marginal cost of each instruction in compressed at the high end of utilization. The first instruction costs a lot, but after you're already running it the best thing you can do for efficiency is take utilization on that machine straight to 100%.
And even if the machine is already on, giving it instructions to run will make it change the CPU frequency (DVFS), access the RAM, etc. which does increase its power usage.