Hacker News Reader

Comment by hsn915 | Hacker News Reader

The fallacy of premature optimization rears its ugly head again.

When Knuth wrote that quote, he meant something very specific with the word optimization: low level micro optimizations. Spending a lot of time trying to get every little ounce of performance from every little CPU instruction.

High level reasonable design decisions are not "optimizations" in this sense at all. You should think of them as non-pessimization.

Choosing Python or another slow language is premature pessimization. You just make your program slower for no reason.

So choosing a language based on how programs written in it perform is simply non-pessimization.

If you are interested in an expansion on this idea, checkout Casey Muratori's lecture about philosophies of optimization: https://www.youtube.com/watch?v=pgoetgxecw8

The article makes claims about software being "fast enough" and not needing further optimization. But what you call "fast enough" is probably 10000 times slower than what it could be.

I'm not joking.

People who program in slow languages have a really skewed perception about performance.

If a page takes 5 seconds to load, they don't see that as a problem.

If a server takes 500ms to respond to a request, they don't see that as a problem.

This is simply unacceptable.

View on HN

I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comments, votes and most of the code I had to write was in django or rq endpoints.

On first load, not only the page is ready, but the video has started to auto play (including buffering), under 3 seconds. Less than 500ms on a reload.

The whole project has been running for 10 years on 7 servers. Hosting costs under $1K/mo.

And even better, the code base is a mere 11K LoC, so a single dev can manage it. And make a living out of it.

That's because:

- Python is almost never the bottleneck in a website

- Python wraps compiled languages for slow stuff. That's the whole point.

- your archi matters much more than the language you chose. Because good db providers, proper caching and optimized data fetching will be the pareto moves, unless you are google size.

- you will not code most things anyway. You are not going to code the video encoding, you will use ffmpeg. You will not recode a database, you will use an existing one. You will not code a server, you will use nginx or something else. What you will spend time on is creating logic for your clients.

So I think your are extrapolating this rare use case where your POV makes sense, and ignore the 99% of the case were, indeed, a slow language doesn't matter.

In fact, if electron success tells anything, it's that while I do value speed a lot, the market has voted.

> I have a half million users a day streaming website coded in Python. It does machine learning, uploading, encoding, dynamic load balancing, live search, comments, votes and most of the code I had to write was in django or rq endpoints.

Sounds much more likely that you wrote in python the code that bundle together the libraries that peform the actual network streaming, machine learning, encoding, search, etc, and that those libraries were not written in python.

Yes, that's my whole point.

Where in the stack do you see that Python performances matter?

Only one software out of a 1000 will actually need it. The others will mostly do plumbing or automation.

And scripting languages excel at both.

If you don't fall in that category, then yes, do use a fast language. If you are coding a chat server, use Erlang. If you want to do a rendering engine, use Rust. But if you most likely want to code your company CRUD software and glue it to some external services and providers, use Python.

> Where in the stack do you see that Python performances matter?

As we just saw, most of the code forming your stack is not written in python because its performance would not allow it.

> Only one software out of a 1000 will actually need it.

Not what I see from your example.

It seems to me that you are overinflating the volume of the python code that you wrote and are blind to the huge volume of performance critical code you just take for granted.

You probably mean: most of the code that most of the programmer write is disposable code assembling reusable libraries. This code performance does not matter therefore python is fine.

I agree on this. The problem I see with this reasoning though is that since python is so slow and error prone compared to compiled alternatives that it is unfit when scaled up from small disposable programs to largish permanent software, in a world where all programs trend to grow and stick in production for longer than expected.

3 seconds to load and $1000/mo hosting costs is way over the unacceptability threshold for me.

I don't know what is the website you're running or how many people use it or how much money you make. I know you can make a lot of money doing stuff in python or electron. Does that mean the market has voted that performance doesn't matter? I don't think so. Jira makes a lot of money but almost everyone hates it. I know a lot of other software that is very slow but still makes tons of money.

> you will not code most things anyway

You say that like it's a good thing! It's a bad thing.

You may say it's because you don't need to. But the flip side is that you can't, even if you wanted to.

You can't program a database or a video encoding or a web server in this language you are using to write your program.

So what is the point of this language?

You can't even use it to cache things in memory. That's why everyone uses redis or something similar.

> When Knuth wrote that quote, he meant something very specific with the word optimization: low level micro optimizations.

Sure, but that isn't to say his advice can't be applied in other contexts. Applying optimizations before knowing where a bottleneck is just a bad idea, regardless of the situation. To constrain the advice to the original context is sort of odd, but maybe there's more to it than that?

I always thought the risk of premature optimization was similar to the lean principle of deferring commitment: you should to wait until the last possible moment to a decision that has high switching costs. In both cases you want to have as much information as possible to taking action.

As an aside, this bugged me for a long time: how _much_ information do you need? Recently I co-presented with the CEO of a successful manufacturing company. At his firm they use the military's rule of thumb of taking action when you have 70% of the information. Deciding sooner than that and you risk making a bad decision; Waiting for more information after 70% and you likely are missing an opportunity.

The advice doesn't generalize to high level "optimizations". The problem with optimizations (low level) is they make the code more complicated and more difficult to understand. This is a problem. You better have really good payout. (It also takes a lot of time to apply micro optimizations to everything, so if you agonize about every CPU instruction you will never get anything done).

But, general high level design decisions that don't produce slow code by default do not suffer from this. They don't make the code any complicated or hard to understand. Quite the opposite, they usually simplify things. Because most of the time, the way to get good performance is to do the minimal thing that needs to be done to perform the task at hand. The code you write will be very straight forward. There's no switching cost.

On the other hand, choosing a slow language has a high switching cost. You can't just take a bad design and try to optimize it. It usually doesn't work. You have to rework the whole design.

EDIT:

I have one other important point to add:

> Applying optimizations before knowing where a bottleneck is just a bad idea, regardless of the situation.

Ignoring any and all performance concerns using this kind of reasoning will result in a codebase that is permeated with slowness. The program is slow but you can't identify any single bottleneck. You end up with a monster that simply can't be fixed.

Worse, you may not even realize how slow it actually is. If you have profiled it and found no particular bottle neck, you may think this simply the limit of computer performance and the task at hand cannot be performed any faster than this.

I love that you can replace "optimization" in your first paragraph with "abstraction" and have the same results. :D

This actually leads me to see the concept of "micro abstractions", and I'm going to see if I see those in hard to deal with programs, now.

I'm not sure what you mean, but with abstraction it's the opposite of what I said.

Premature abstraction really is very very bad (I won't go so far as to say it's the root of all evil).

The higher level the abstraction, the worse it is.

Unless it's born from experience writing the same kind of program many times. In which case it's not "premature".

"micro abstractions" are actually fine. You notice a recurring pattern in the code so you "abstract" it away into a function.

My guess is we are talking of different things when envisioning micro abstractions.

I have in mind the projects that went from a handful of data structures and functions to dozens of each. Typically in the vein of abstracting away whatever immediate problem was faced in the code.

Have a workflow? Make a workflow abstraction that can run any work.

Have code that needs to retry? Make a retry facility that can retry for any reason. Than make a grammar to specify what backoff should be used.

While doing those, realize you want a rules engine to determine what work and what backoff to use.

Then, put in a sat solver. To determine if a plan can be done.

...

None of these are really bad things, in and off themselves. Odds are high they would all be poorly done, or themselves a giant undertaking.

> the lean principle of deferring commitment: you should to wait until the last possible moment to a decision that has high switching costs.

Last responsible moment, not last possible, moment. That's a big difference. You shouldn't start on the work at the last minute, that's too late. But you also don't (necessarily) need to start on something 2 years before it's due (unless it's big and/or there are anticipated problems that will need to be addressed that the time would permit; that's what makes it the last responsible moment).

Agreed, I should clarify that in my original comment.

How do you know when you've had 7/10ths of "the information"?

There's a great deal of scientific literature about this topic, BTW. Especially in machine learning.

> Sure, but that isn't to say his advice can't be applied in other contexts. Applying optimizations before knowing where a bottleneck is just a bad idea, regardless of the situation. To constrain the advice to the original context is sort of odd, but maybe there's more to it than that?

Donald Knuth writes entire volumes of books in assembly language, because the details matter. If you want to have that point, then you probably shouldn't be quoting Donald Knuth.

Donald Knuth is the guy who points out that "for(int i=size-1; i>=0; i--)" is faster than "for(int i=0; i<size; i++)". When he says "don't sweat the small stuff", its because his books are filled with _incredibly_, detailed stuff. Far more detail than any other programming book / algorithms book.

IIRC, the "premature optimization is the source of evil" statement was about how GOTO statements and how efficient they were. The __context__ is that GOTO is indeed a very fast construct (especially in the context of replacing recursion).

He made the comment a few times, apparently, in the context of GOTO it was this:

> Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%.

From: "Structured Programming with Goto Statements", emphasis mine.

Archive link: https://web.archive.org/web/20131023061601/http://cs.sjsu.ed... (I haven't read it in a while, and it's 41 pages, so I probably won't get to it today.)

If one cared that much about speed, they'd use `--i`, not `i--`

Knuth wrote that discussion point in assembly language.

I translated it to C so that people got the gist of it.

Per godbolt with GCC 11 and clang 13, there is no difference between those two. Both end up compiled to the same machine code (though GCC and clang both compile them differently, both are consistent with themselves with respect to pre or post decrement).

https://godbolt.org/z/qbEo9GzEx

> The article makes claims about software being "fast enough" and not needing further optimization. But what you call "fast enough" is probably 10000 times slower than what it could be.

As a fun project, I started writing some code to do calculations on matrices. Wrote it in Python and used Numpy. Then I wrote it in Julia. Almost immediately it was a 100x improvement. I spent time doing "premature optimization" and netted a 10x improvement. So overall it went from minutes to milliseconds.

I guess if you're writing a website or something then I wouldn't worry, but for other things, doing optimization from the get-go can save you a ton of time and effort later on. And really, such optimizations can affect the architecture in ways you wouldn't think of without it. Reworking your architecture down the road is a huge time sink. I like to make sure mine is close to where I want it to be in the beginning.

> I guess if you're writing a website or something then I wouldn't worry

Despite popular sentiment on HN and Reddit, a website is not a piece of software that can tolerate slow code. Unless you are ok with your website dying when 100 people visit it at the same time, or otherwise investing in a complicated "cloud" infrastructure to make it possible to have more than 100 concurrent visitors.

From my experience most of your assertions have a hidden assumption that the software requirements are complete and they are good. Computing something 10000 times faster is useless if it turns out to be the wrong computation. Furthermore, things that run 10000 times faster than other things tend to be harder to adapt and change.

Requirements that are good enough to warrant a 5 second page load time being "unacceptable" must have, at some point, been prototyped with real use cases, real data, and in real environments.

Getting to that point - where the business case is connected with such a strong through-line to the user experience can literally take years. Early in a project lifecycle it's more important to design software that is easy to change than software that is fast.

Language choice does not matter, performance does not matter; the design philosophy of the developer(s) when building something new is everything. That philosophy should be "I'll likely have to change this" not "I need to make sure this runs quickly". Any high level design decisions should be made with the former in mind, not the latter.

Once you've earned that product wisdom, sure - carve out the slow thing and write it in CUDA or something. If things are easy to change, that's not a scary proposition; it's just part of the natural project lifecycle and Knuth lives to die another day.

This is a false dichotomy.

"We either have perfect information or we make software that is very slow".

It is not an either/or situation - it's a maturation process. Make flexible slow software first, then optimize the parts that need to be fast.

Optimize iteration frequency of features not for loops.

What I'm advocating here is not for optimized code. Only for non-pessimized by default code.

This kind of code does not tend to be rigid as you might imagine. Quite the opposite. Since it doesn't introduce a ton of premature abstractions and stays straight to the point, it tends to be flexible and easy to change/adapt.

You are still thinking in terms of a false dichotomy:

- Flexible but slow

- Fast but rigid

But it's a false dichotomy.

You don't need optimizations to get high performance software. Computers are very fast. You just need non-pessimized code.

It's funny. People use "computer are very fast" to dismiss all performance concerns. But this is not how we should think about it.

"Computer are very fast" means we don't need to optimize (as in micro-optimize on the instruction level) code to get high performance. We just need to get the clowns out of the car.

I think we're largely in agreement here e.g. I agree 100% that premature abstractions hold a lot of inertia. I'm just advocating for making decisions that optimize for "the right thing" over speed alone. In my experience the right thing tends to be adaptation, but that's because I do a lot of R&D work.

Sometimes yes, you can eat your cake and have it too w/r/t performance and flexibility; but the choice of programming language cannot be categorically relegated to "always pick the fastest one". There are no silver bullets, there will always be tradeoffs.

Flexibility matters, ecosystem matters, developer experience matters, third party libraries matter, good requirements (or lack thereof) matter, budget matters, timeline matters, security matters, and yes, performance also matters - it's just not the only consideration.

High level design decisions, language included, are not "pessimized by default" because they deem some of those other considerations more important than speed.

OTH not choosing Python for tasks where the language performance really doesn't matter at all (for instance shell scripting stuff) would also be futile. It's simply a matter of using the right tool for a specific job (and in Python's case, that 'tool' isn't even the language, but the batteries-included stdlib).

I don't buy into this argument either.

Using python for shell scripting requires extensive knowledge about the language and the standard library and all the gotchas that you can fall into.

This sort of time investment does not pay off if you just want to write small scripts that perform simple tasks.

If the task is simple, then a bash script would be better.

If the task is not so simple, then a language you are really familar with is better.

If you accept my premise about how programming in slow languages is generally a bad idea, you would not spend that much time learning any such language with all its ins and outs and gotchas.

For background, I personally have spent a lot of time learning python, and only later in life have a realizd what a mistake that was. Now I only use Go for server side programming. I'm also using it for command-line programs. Such programs usually start small but grow over time. Choosing Python for them would have been a costly mistake.

> If the task is simple, then a bash script would be better.

Bash script don't work on the Windows command line, and bash relies on UNIX tools as its "standard library", which also are not available on the Windows command line. That's the whole point really, Python has very good cross-platform support (and in this case cross-platform doesn't just mean "Linux and macOS", but also Windows).

> If the task is not so simple, then a language you are really familar with is better.

IMHO every programmer should be familiar with at least a handful programming languages.

Bash can and does work on windows if you install it. Just like Python.

Also doesn't windows now ship with a linux compatibility layer anyway (WSL)?

EDIT:

I just realized you are the guy who wrote sokol!

If your main work station is windows and you don't work on linux, then replace bash from my previous comment with bat(ch).

Bash is not the standard shell on Windows, and it also relies very much on UNIX-style filesystem conventions that don't map very well to the Windows environment. I wouldn't ask my users to install PowerShell on Linux or macOS either.

Asking Windows users to run Bash scripts is essentially the same as asking them to build a C/C++ project that uses Autotools and Makefiles. It may work with lots of tinkering, but it's a massive world of pain.

> If your main work station is windows and you don't work on linux, then replace bash from my previous comment with bat(ch).

I usually switch between macOS, Linux and Windows at least several times a week, sometimes several times a day, maintaining different scripts to do the same thing wouldn't make sense.

> Bash is not the standard shell on Windows, and it also relies very much on UNIX-style filesystem conventions that don't map very well to the Windows environment.

Windows was built by people who started by copying from Unix. This is part of the reason Bash works fine on Windows (ie What is your problem with it "mapping" to things?). You can pipe commands, you can stream text, you can edit binaries. There are things that slow down (or can't be done, modify permissions easily, etc) because of Windows tradeoffs, not because it's fundamentally different. An OS does OS things and a shell talks to the OS.

Just write in the header of the script: "This script requires Fedora Linux 33 or higher" and forget about outdated OS'es, unless it's your paid job to supported them.

If you OS vendor refuses to supply you with appropriate tools to do your job, then just chose another vendor. Vote with your foots.

Talking about gotchas and then recommend bash is not a sign of credibility in my book.

Bash has one strong point: 4 to 5 lines admin scripts for which concision and expressiveness matter more than anything else. That's it.

There is nothing else it can be better at than ruby, python, perl, php, or even go or kotlin.

The argument you need to learn x stands for bash as well, btw.

I only said to use bash for small scripts. Basically just a series of commands. Maybe setting up one or two environment variables. Anything more complex or involving "if" I would almost never write in bash.

OK but shell is cuter than python.

100% this.

It's now right up there with "X considered harmful", which 80% of the time translates to "I don't like X".

Specific to Python from the article, I'll add that I personally am done with dynamically typed languages for anything where more than one person needs to work on it or it needs to be long-lived. Static typing is simply too useful. Dynamic typing is (IMHO) a false economy. I've often said "Python means writing unit tests for spelling mistakes". Refactoring a dynamically typed code base of any reasonable size is a comparative nightmare.

It's same story again and again.

Shell good for simple scripts. — Shell scripts are pain to support!

Perl is astonishingly good for text manipulation. — Perl is write-only language!

Python is easy to learn and start coding. — Python2 to Python3 migration is decades long hell!

JavaScript is the language of the World Wide Web. — JavaScript is nightmare to refactor!

And so on.

If language easily accepts anything that developer pops out of his head, then program will be hard to support and refactor.

If language limits developer to certain good, safe, performant solutions, then it hard to write programs in it, because you will need to learn good patterns first, which may take years.

> hard to write programs in it, because you will need to learn good patterns first, which may take years.

People really underestimate the time it takes for devs to figure this out. My company has been on TypeScript for over three years now and it's a giant mess. Everything is arbitrarily typed with no design philosophy or organization behind anything. Which leads to these massive interfaces with every single field being optional. Which means you're still doing run-time checking all over the place.

I've always maintained that if your team sucks at organization then no amount of tooling will save them. You're not so much working as a team but as bitter enemies. Possibly the turnover is too high in the company. You have mercenaries rather than ideologues. Not that I'd want to work with a team of pure zealots. But there should be enough people holding the architecture in place. It helps morale as well. You don't want to work on a project that no one gives a shit about.

In the current age of software, we spend way less time designing with any future thought whatsoever. In that respect, I concur - the article's position is damaging.

You can design slow crap in any language, but I'd rather use something that has the potential to be robust and fast for my problem domain so I don't have to rewrite the entire app in the future to language "X", when I could have easily started with "X".

Make no mistake - if you do this, you made an upfront error in selecting your language for the business problem. It may not be an unrecoverable error, but it is a failure in judgement regardless.

Yeah, it seems like he was referring more to bike-shedding than anything. Focus on the efficient, meta-goal, and make efficient choices for the highest value.

In this case, since the entirety of the project is to be written in a specific language, it's usually worthy of placing that in the more macro-category - where that choice affects all other decisions.

Don't spend 4 hours of your time on a `for` loop but half an hour of your time deciding on the language and architecture.

My favorite aspect of Knuth's take, is that he definitely concerns himself with the low level aspects of programming when writing things.

I don't mean that as a criticism, either. I keep meaning to strive to having the level of cognizance to memory usage that he just drips into his software.

> High level reasonable design decisions

Viewing language choice as a high level design decision for a project of nontrivial scale is a symptom of bad high-level design decisions.

Reply on news.ycombinator.com