Premature optimization caught up with me as well
martin.leyrer.priv.at
martin.leyrer.priv.at
It is best explained by Donald Knuth himself. The full quote is:
> "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified. It is often a mistake to make a priori judgments about what parts of a program are really critical, since the universal experience of programmers who have been using measurement tools has been that their intuitive guesses fail.".
Source: Structured Programming with go to Statements by Donald E. Knuth. http://web.archive.org/web/20130731202547/http://pplab.snu.a...
All you need is the measure part. Then you won't do the evil thing of optimizing something that makes it more brittle and hard to maintain etc.
A large part of that is due to the immaturity of software as a profession relative to other professions. Measuring things takes tremendous effort and many developers don’t want to challenge their deeply held assumptions of how things work. Often when you present performance measures to developers they will invent an excuse to throw the data out if they don’t like the conclusion.
Better training and better tools can help make obvious their mistakes but I don't know how to combat the dogma. I've often been shocked how almost superstitious developers are - you'd think developers would have a more scientific approach but they are just as likely to chase after ghosts than optimize what needs to be optimized.
Seriously though, there are plenty of developers who can cautiously evaluate performance as they go. It's just that it's not a super common skill (nor is writing low level code, etc), but it exists and is happening all the time. Game devs for example.
I think by itself the phrase is taken to mean to keep the code high level and abstract as long as it might still need to change, and then only optimize once it's overall structure is established, because once you start optimizing for performance you are carving stone and destroying flexibility.
But the full context shows that the word "premature" in this context means to say that it's premature to optimize before you even know what to optimize.
People that have been in the performance business a long time, the kind of people you want if performance matters, can estimate the performance of code without running it to a degree of accuracy that is surprising to people with no performance engineering experience. Not understanding the relationship between code and performance is not the same as no one being able to understand the relationship between code and performance. It is a specialized discipline but people in it don't need to measure code for macro-optimization purposes. You still need to measure some things for micro-optimization purposes, but even there you'll find savants that really don't need to.
Naturally, there is an element of many developers overestimating their ability to estimate code performance. That is not universal though. I've worked with developers that could consistently and precisely describe the performance characteristics of code on real hardware without running it. A lot of computer science is premised on this being possible. Acting like this is unknowable is doing a disservice to computer science as a discipline.
If you understand how code interacts with hardware then performance is deterministic and obvious. There are degrees, of course, but some developers have a deeply intuitive understanding of these relationships.
This conclusion is strange to me. Literally every performance improvement I’ve seen is done at the end of a project. Of course you try to avoid architectural defects, but software is not immutable. So you punt the performance work later in the cycle once a) you’ve verified the approach overall roughly works b) you’re confident you have a handle on the business needs c) you have identified a need to improve performance. Any performance defects that turn out to be architectural in nature are still fixable, just more expensive (hence the desire to fix the most obvious ones). For example, look at any piece of Google infrastructure. They always go through several iterations because it doesn’t matter how good your intuition may be. You can still be wrong and the landscape isn’t static so whatever was right at the time may have evolved to not be as right. For example, protobufs made the right choice about encoding for space when bandwidth was more limited. Now that bandwidth is much more free and compression is more effective, it’s better to use compressors instead of trying to find a more optimal binary coding.
If performance is awful to begin with you can improve it visibly by fixing a few hot-spots (like unnecessary O(N^2) for N which can be large). But when I do profiling for software which is not too bad to begin with there usually no single place which can be changed to give statistically significant improvement - there are many comparably slow places. One can try to improve them piece by piece but sooner or later abstractions will get in the way and if performance is still not enough you'll need to start from scratch keeping performance in mind.
And that's the crux of it. Donald neglected to understand that most programmers will takeaway "optimization is bad" and all the subtleties of his point will be lost.
Then we get things like electron (ducks for cover)
That writing was 1974, Lisp had already existed.
We get electron because it's a batteries included (and remote and TV) cross platform GUI.
And has a huge talent pool, and acceptable performance for most users.
Back to ”Premature optimization is the root of all evil”: I think that’s a bad statement. The only thing that says something there is all, and it is incorrect.
“Premature (noun) is the root of evil” is true for about every noun you can out there. “Premature design is the root of evil”, “Premature programming is the root of evil”, “Premature debugging is the root of evil”, etc.
Adding all implies there aren’t other things that are (in this context) roots of evil. I don’t think that’s true. As an example, one can spend excessive amounts of effort thinking about, or worrying about the ‘beauty’ of source code.
But on code optimization, k deeply nested loops are not a bad guide to O(n^k), and a candidate for optimization.
That said, I once cached something that obviously needed to be cached - so obvious in fact that a deeper level in the system already cached it.
It actually is (give or take for translation) "For the love of money is a root of all kinds of evil."
Timothy 6:10
The love...
A root...
All kinds...
That is not premature optimisation. That's called plain old jumping to conclusions. Next.
Average quality of developer is way lower than it used to be (people aren't even capable of optimization). The need for optimization is also not as high: hardware is fast and cheap, abundance of Electron apps illustrates that.
I have a feeling that today is the age of premature abstractions and premature scalability.
The main point is that there's the right and the wrong time to do anything: optimization, scaling, testing, monitoring, etc. When a concept is deemed absolutely positive, naturally it will be applied prematurely.
I would disagree with you only that I'm not familiar with a time when premature monitoring was added that caused me a lot of pain. If anything, the opposite has been true. Can you explain what you mean there?
There's a lot of examples where you can catch something with monitoring, but it doesn't necessarily mean that you should.
A recent one from my memory: in a SaaS product a team shipped a bug that went unnoticed for a few days. It was feature flagged, so it only affected a small fraction of customers and didn't trigger any global alerts. Now, since it didn't trigger alerts, the natural post-mortem action plan was "better monitoring". That would mean monitoring and alerting on "rate of errors by customer" (or "rate of errors by endpoint by customer", I don't remember).
Given the usage pattern of the product, it was impossible to create a global monitor like that, we'd have manually configure it for each customer (and we had thousands of those). And even then, we'd inevitably be dealing with false positives every week.
The right action plan was to learn from failure, but do nothing. We got extremely unlucky during infrastructure update, shit happens. We don't need to build a complex monitoring system that catches one bug every 5 years.
Snarky, but in my experience 6 sigma true.
which are optimizations.
> Average quality of developer is way lower than it used to be
I think this is an illusion based on the fact that the number of software developers is growing quickly, which means that most of them are new & young at all times, giving the misleading impression that things are trending down from the perspective of someone experienced. In absolute numbers, there might also seem to be a lot more unserious developers today than 10 or 20 or 30 years ago, but there’s also many many more serious and extremely good devs too, and that can be harder to see from where you sit. Schools have gotten better in many (but not all) ways too.
> The need for optimization is also not as high
With Moore’s law slowing down, and with competition in software greater than ever before, and with improvements to compute continuing to outpace improvements in memory bandwidth, I think the complete opposite is true, that the need for optimization has never been higher and is increasing as we speak. And I think in practice more people are optimizing systems like crazy compared to when I was younger. In school we learned about the occasional writing of a small loop in assembly language. Today we have cloud caches, and GPUs running CUDA code, and much better compilers. You might just not be getting exposure to optimization where you work, and there are places where people don’t see a lot of it, but there are tons of groups and companies where people are always optimizing. I see a lot of it, and I don’t think Electron apps demonstrate it’s not happening. Electron exists and is popular because it’s easy, and that’s a valid development choice for a lot of people. It might be slow, but there have always been frameworks that are easy to use that are slow. And then there have always been the people who care about performance and avoid slow frameworks, they’re still here and growing in number.
No one wrote software in the 1990s like the advanced software we write today, it is qualitatively different. I think that perspective is lost on a lot of people that are not as old as I am. That increase in complexity is a serious barrier to understanding. The state-of-the-art today requires cognitive abilities that are much higher than they used to be. Hell, it is why C used to be an adequate language for most tasks.
a. What features are going to be added over the course of your application b. What features are going to be popular c. How these features scale, independently of the overall application
Architecting for reasonable performance should always be a goal. Architecting for scalability, however, is often wishful thinking that from my experiences has lead to more harm than good.
Not true today.
Head down some optimisation path instead of just getting the damn thing to work - that could sink the project.
Today - speed is a feature.
Speed has always been a critical element.
Premature optimization is when you optimize without a slowdown anyone has noticed.
Going overboard with architecture is designing for scale when you have had no need to scale.
But does the fact that a business will push you to keep delivering new features mean that it's often better to prematurely optimize, prematurely architect, so that you don't end up stuck with code that you hate and can't maintain?
When something takes 3 minutes we're beyond talking about optimization. Something that takes 3 minutes is usually a sure sign of really really bad architecture, unless we're talking about time intensive tasks like video encoding or downloading a huge amount of data. And even then you need to question: do we need to download this much data?
Every program that has any need for optimization will quickly turn into a death by a thousand cuts.
There is no right or wrong level of premature optimization, it all comes down to how much time it takes to code it to be more performant. if it takes a tiny amount of time, do so, even if you get little gains. if it takes twice as long to make it more performant, only do so if necessary or results in more than 50% gains.