I have optimized websites based on a basic understanding of how CPU's work before. For example it's much faster to do 200 checks on an object then load the next object vs doing one check at a time for each object and thus reloading objects 200 times. This ended up being a 30x or so speed up and seemed like magic to half the room.

It's not about knowing the minute details so much as understanding what's going on well enough to model it in your head.

PS: Assuming you are operating on lots of data, a small scale test can and did go the other way.