The Mature Optimization Handbook
facebook.com
facebook.com
[PDF] http://carlos.bueno.org/optimization/mature-optimization.pdf
[Mobi (Kindle)] http://carlos.bueno.org/optimization/mature-optimization.mob...
[ePub] http://carlos.bueno.org/optimization/mature-optimization.epu...
The first few chapters in particular are a nice introduction to reasoning about performance, and the availability in multiple formats is also nice!
I wouldn't say that they're necessarily overkill or out of reach, but things to keep in mind as you're designing and building your software. Making sure that you can measure and instrument your code can save a lot of grief later on, even if it doesn't make sense to be doing those measurements right now. The flowchart is absolutely true: You can't prove that it's fast without measuring it.
I believe that by presenting such a view [on the proper use of goto statements] I
am not in fact disagreeing sharply with
Dijkstra's ideas, since he recently wrote the
following: "Please don't fall into the trap of
believing that I am terribly dogmatical
about [the go to statement]. I have the
uncomfortable feeling that others are making
a religion out of it, as if the conceptual
problems of programming could be solved by
a single trick, by a simple form of coding
discipline!" [29]. In other words, it, seems
that fanatical advocates of the New Pro-
gramming are going overboard in their strict
enforcement of morality and purity in
programs.The group had a very effective technique for estimating depth from stereo camera systems called PMF. Now this was an edge based algorithm, that used a lot of 'high level' (at the time) data constructs; you do Canny, calibrate an align to the epi-polar geometry, search and compare edges under various criteria, create lists of matches, etc. It was accurate, which is what mattered but slow. Now one smart guy in the group worked out that you do achieve the same outcome using a very different approach (known as stretch correlation) to achieve a similar result that on the right hardware would run much quicker. Well to cut a long story short the algorithm got written (worked well) and then someone went off and designed and built the specific hardware, including a chip optimized to run the algorithm. Frankly it was incredible it worked considering this was a couple of PhD students and our access to fabrication was, well typically British shall we say, which lead to yield problems. However, by the time it was all up and running we had moved a lot of development onto Linux on some nice shiny new Pentium PCs (133MHz!) which had little trouble running the original PMF algorithm almost as quickly and more stably than the purpose built chip.
Now in some respects this was all fine. We were a University whose purpose was to teach and some smart people learnt how to design chips and develop algorithms, all good. However, the intention was also to create something of practical value and in this it failed. It failed because people forgot Moore's law and that today's optimized code is tomorrows cranky old, cryptic ball of mud.
I apologise if I'm just reminiscing here but I wanted to share.
Is it possible to share some of the details of "postmortems" of large-scale systems in more detail? This would really help academics who are interested in working on techniques to diagnose such issues. Perhaps log files, history of monitored metrics, etc.?
thanks,
Restricting your variables is important, because you can't be sure exactly which change resulted in the improvement if you made a dozen changes(which is why threading is sometimes dangerous, because, like jwz's famous regex quote, it tends to turn one problem into many).
You're going to see things that don't make any sense, even if your gut says differently. This point is hammered home quite a bit. I've seen a case where removing what I thought was dead code reduced performance by a measurable amount... I still can't explain that one.
I haven't read the book yet, but I believe the key to a mature optimization is knowing clearly what needs to be optimized. There is no way to know it if you don't measure it. But the thing is that you may be measuring the wrong thing. If you focus on a class, for instance, without the entire context, you may be trying to speed up the wrong piece of your application. Make sure you measure the entire flow, from the user clicking something to him seeing the response. Sometimes you will find that what needs to be optimized is not even your code.
"How a program works and how it performs are very different things." -- Yes, yes... That's also the case for many random x and y.
"By all means, use your brain and your training to guide you; it’s good to make predictions. But not all predictions are good." I didn't find a translation for "La chèvre et le chou", which means keep the goat and the cabbage and is ironic. So yes, predictions are sometimes good sometimes not that good. A very many random x also comply to this proposition.
And so ooyawnnn.
But to have a little positive note, some quotes are fun, I actually enjoyed this troubling Mise-en-abîme[1] "I once generalized from a single data point, and I’ll never do that again!"
[1] Not lucky today, it seems to not have any English translation either, interested reader can wikipedia it or tell me wrong.
It's hard to remember to practice those platitudes day-to-day, which makes them worth repeating. You must register your predictions beforehand in order to avoid hindsight bias. You also have to avoid getting attached to them. Programmers very often commit map-v-territory mistakes, asserting how a thing works as if it were an observation of how it performs, and so on.
If all this is old hat to you, well, congrats! But it's not to everyone. And keep reading. It gets more practical after Ch 2 or 3.
Hilarious!