But in general, unrolling loops, caching (some) results, and eliding redundant code rightfully is the compiler's job. That's a different kind of speedup.
I pretty strongly disagree that with the meta-parent that it's the "compiler's job to speed up your code". It just doesn't have enough context to be able to do it effectively. I would actually prefer a predictable compiler to one that tries to do all sorts of magic tricks that can be perturbed by seemingly trivial code changes.
That's not what jordigh is talking about. Let's say you figured out a clever way to get your problem to O(log n). That's definitely your job. But in practice, constants matter - that is, 10 log n is way, way worse in practice than 2 log n, even though they are both O(log n). And that's the kind of optimization that compilers may be better at than you. Once you have convinced yourself that the performance of some component matters, a good principle is to figure out the best complexity class you can (using the "right" data structures and algorithms), then write the most idiomatic code possible.