Do you know how much your computer can do in a second?

It's because the extra slowness is supposed to be buying us software thats safer and easier to write. The argument has never been about how fast can we be, it's about being fast enough where the cost of talent completely annihilates the cost of compute.

If I were writing some python, which means I've already made that trade off, you wouldn't much like my answer if you started asking me such questions. Which is a shame because I'd probably rather have quite a lot of respect for the same point if we were working lower down the stack.

> Alternatively: do you know how much your computer could do in a second, but isn't, because the majority of software is so full of inefficiency?

Amusingly, in the first example, the compiler's static analysis concludes that the entire for loop can be omitted. I have no idea how they came up with that precise number...

Agree 100%.

As a hobby I like to hack 6502 assembly language on original hardware, I have Beebs, Ataris, VIC20 and C=64. 2Mhz is the limit. It is amazing to modern eyes how much computation on these systems you can get done once you strip away all the layers of abstraction and indirection modern systems are saddled with.

The situation gets even more extreme when you consider graphics cards, and their ability to do thousands of operations per screen pixel per frame.

I like your Mac Plus article; that reminds us that the user experience has not got necessarily got better with all this processing power.

Some of the work of Distributed.net ( http://www.distributed.net/Main_Page ) is wonderful. Does anyone know if this idea could be more than it is currently, where computers (more than ever) are sitting idly and not contributing their cycles in any meaningful way? Even 5 minutes of raw CPU 100% usage per device could do some serious computation. Theoretically superseding modern supercomputers.

Is it efficient if it takes so long to get to market that the product is irrelevant by the time it gets there?

Is it efficient if the product is buggy and insecure because it's inherently more difficult to program in assembly compared to Python or Java?

Is it efficient if the product is not extensible and inflexible because every engineering decision went towards efficiency, which is inherently opposed to flexibility? (Early binding is inherently more efficient than late binding for all the same reasons it is inherently less flexible.)

Or do you insist on defining "efficiency" solely to mean "machine efficiency" instead of thinking like an engineer and understanding the concept of trade-offs where no decision is inherently worse than any other?

True

It's amazing how somethings are extremely inefficient, and even worse, nobody questions themselves why this is like this

Most of 'this is taking too long' is because of IO, still, sometimes it's done in a very inefficient way as well

C# is a pretty good example of where the tradeoff between speed and safety can be a good choice.

I wrote a non-trivial stock trading app in C#, and for sake of this conversation I added some loop counting. it analyses about 1.5million ticks per-second (against trade rules). This is a single thread running on a 3.2ghz core i5, no I/O.

That's exactly what got me interested in learning x86 assembly language. It's pretty neat to see how fast a generic loop can be when the entire executable is <10 bytes.

The inefficiency of computing platforms is like the amount of congestion. Both expand to the level of being just barely bearable.

If a computer is too slow nobody will use it; if it's only irritatingly slow, some will use it and; if it's slow but still bearable enough, everybody grumbled but still uses it. Development costs and building programs with higher level languages and using big, high-level frameworks saves in development but increases the cost of using the system. The result is that some majority of programs will always be optimized only to the point where they are barely usable.

Similarly, when traffic is always as congested as the people on roads can bear: those will drop out who really hate congestion or prefer to do their trip at another trip but they will be replaced by people who are willing to pay the time tax again. When space is freed on the highway or arterial road, some people will figure out it's worth the trouble and head out.

This means you can't make the user experience faster by getting faster computers and you can't build more roads and lanes to get rid of congestion.

But there's more.

Because a large mass of congested traffic is more difficult to separate and channel into exits congestion on large roads tends to be worse. If you have a village main street with one lane each way on a road, and that road gets traffic congestion that's much less total congestion, area-wise, and it will clear up pretty quickly once the head of traffic is able to exit somewhere. Even so with a grid of 1+1 lane streets, like in a small town. Because the small road gets quickly congested it will only swallow a limited number of cars which will keep a bound on the maximum amount of congested area. Not so with a 10+10 lane highway.

Similarly to computer programs. Slow programs in the early times when everything was simpler could be made faster with sufficient effort. The programs might still struggle in some parts but you could at least invest in the effort to streamline the core operations and make the worst things fast enough. In contrast, consider an enterprise Java program that depends on dozens or hundreds of libraries that amount to a running image of the size of half-a to a whole gigabyte. It takes ten seconds to start and opening each window takes seconds because there's slack everywhere. Everything runs top of something else and even simple operations involve running mundate pieces of code for seconds in wallclock time. Because the computer that's fast enough to run the program in the first place also has the computing power to run a lot of crap in a similar manner. So the amount of crap just increases as the computing power increases, and getting rid of that crap becomes an unsurmountable task. So it becomes close to impossible to make such a big program run fast because you would have to shrink everything between the application and the hardware in half or in one tenth to get some sort of speed. Coincidentally, old computer systems often had faster user interface latency and responded to user inputs in a very timely manner. They could do the cheap things really fast even if the total lack of computer power still made the programs spend seconds on the heavy processing.

What I'd like to know is how this plays out at the operating system level. We can run cachegrind on a program, but it doesn't take into account the effects of task switching of the OS (as far as I know).

I think it would be lovely in these multicore days if we could dedicate a core to a process and have as close to 0 task switching as possible on that core.

>Many of them will have some idea about theoretical time complexity, but then see nothing wrong with what should be a very trivial operation taking several seconds of CPU time on a modern computer.

In my experience developers who over-optimize, or who optimize with disgusting hacks, are a vastly bigger problem than developers who don't optimize at all.

There are a vast number of problems where a several second wait for the user is almost irrelevant compared to the damage done to the code base by optimizing those few seconds away.

There was an article on how much can your computer do 400 cycles (a cache miss) recently.

I have noticed that the size of L2 caches hasn't increased anywhere near the same rate as other memory capacity.

http://www.slideshare.net/EmanWebDev/pitfalls-of-object-orie...

LAME. What you want is a javascript heavy web UI driven by nodejs server, all running on same underpowered embedded platform. At least 5 levels of abstraction above decency, because hey, we can iterate quickly and hardware is cheap!!1

Python's JSON parser will obviously create Python objects as its output. There is a limit of how much you can gain with clever C string parsing when you still have to create a PyObject* for every item that you parsed. Because of this, I don't think you can gain 1000x performance with C optimizations unless the parser is really horrible (unlikely, considering the widespread use of JSON).

Well, I'm not a python programmer, and I guessed 14/18 correctly. Some of the questions are hard to know without knowing details of the libraries - JSON, as you point out, for instance (it's also one I got wrong, expecting better throughput). But since each question lists the actual performance numbers after you answer, you can get a feel for how big the overhead is in trivial pure python.

But yeah, some of the questions were poorly chosen. For example, on my machine, using gcc -O2 as the author specifies, the first program always executes instantly, since gcc optmizes that loop to nothing. That leads me to believe he may be running a mac (IIRC those come with absurdly out of date gcc versions as a result of the GPLv3 dispute?) or something else fishy is going on.

One interesting thing to note is that the memory access latency on his machine is almost an order of magnitude faster than you might expect based on "Latency Numbers Every Programmer Should Know" (https://gist.github.com/jboner/2841832) - and that's not a coincidence; those 100ns have always been a bit conservative, and the latency number has slightly improved in the past two decades.

Related: https://github.com/computers-are-fast/computers-are-fast.git...

You're completely right!

Other readers, check it out for yourself with

gcc -g sum.c ; echo 'disas main' | gdb ./a.out

gcc -g -O2 sum.c ; echo 'disas main' | gdb ./a.out

If the function returned s (the sum number), would it not optimize it away?

"Premature optimization is the root of all evil," _is_ true, the active word here being "premature." Once something is working (and hopefully testable), one shouldn't hesitate to dig in and optimize anything and everything. This is not sinful!

Thank god they're stupid, can you imagine a smart robot that can think a billion times faster than us? It takes us a whole life to generate new knowledge (PhD) but it would take them just seconds. Now imagine all that knowledge accumulated in a couple of days, a week or a month. The last century alone has brought us exponential discoveries with all the technological advancements on our side.

No, we can't even comprehend.

Not related to performance, but please let's add "on machines connected with average DSL speeds and sporting medium-resolution screens."

But the basics are pretty much the same, no matter if it is python or assembly: does this one-liner run entirely or mostly in L1 cache, or does it have to wait for RAM access repeatedly? Does it have to wait for disk or does it have to wait for network? Repeatedly? People who fail at this won't be able to understand the difference between situations where python or not doesn't matter much and those where it does.

Understanding that "computers are fast" (even in python!) is a very important step towards understanding where we make them slow and whether that is because of waste or because the task is naturally expensive.

Based on your skepticism i assume that you just haven't had much exposure to people who are really bad at these thing, despite having all the formal education (and the paycheck to match). "I'm working in ${absurdly high level language}, of course i'm not supposed to care for performance" is what they tell you before venturing off to make a perfectly avoidable performance blunder that would be crippling even in fully vectorized assembly, followed by a few days spent transforming all their code a different, but perfectly equivalent syntactic representation that looks a bit faster.

Not really. You can write Python fast and slow, so if you do write Python code, you can perfectly well care about performance. Learning C takes months to years while adapting my Python code takes minutes to days.

The implementation of true also makes a difference! Not quite an order of magnitude difference, though (except for the shell builtin).

    method           Hz comment
    --------------------------------------------------------
    empty file      500 an empty file gets passed to /bin/sh
    dynamic libc   1000 "int main { return 0; }" -> gcc
    static libc    1500 the same, but with "gcc -static"
    assembly       2000 see below
    bash builtin 150000 avoids hitting the kernel or filesystem

The empty file is the "traditional" implementation of true on Unix.

The assembly solution was my attempt at doing the littlest amount possible, because libc initialization still takes time:

    .globl _start
    _start:
    	movl $1, %eax 		# %eax = SYS_exit
    	xorl %ebx, %ebx		# %ebx = 0 (exit status)
    	int $0x80

Processes and files on Windows are heavyweight. I recall once speeding up a script by about 100 times by changing something of the form

    a >>$LOG
    b >>$LOG
    ⋮
    c >>$LOG

to

    {
      a
      b
      ⋮
      c
    } >>$LOG

14 / 18 and I don't really program python (e.g. have no idea what the bcrypt lib's defaults in python are...) - but performance is something I've always cared about, and most of these are things you might happen to know.

I'm surprised by the poor memory performance in his tests; my machine get's around an order of magnitude better performance in terms of throughput; which leads me to believe he's compiling using a very outdated gcc, and/or has really slow memory (laptops- you never know), and/or (reasonable, since he only mentioned -O2, but depends on the bitness of the compiler) he's compiling in "compatibility with 80386" mode.

I think it's odd that people still haven't quite figured that one out yet. People use "-O2" all over the place, when that's rarely faster than "-O3", and they leave out one of the simplest optimization options the compiler has - "-march=native".

Ya, I'm pretty proficient with MySQL and was surprised to get both of the SQL questions wrong (although close), while getting 13 / 18 overall. For the second SQL test, I expect that the second of the two iterations it managed took significantly less time than the first, since the data should have been cached. I guessed 100 for that one, thinking that it would probably manage more than 10 and 100 would give me more leeway - shouldn't have over-thought it.

Assuming your pattern is static, it shouldn't be much slower. String matching can be done in linear time with some preprocessing: check out Knuth-Morris-Pratt and Boyer-Moore algorithms.

Basically, the idea is that you build a deterministic finite state automaton and try feeding the string through it. Each character would cause exactly one automaton transition. Therefore, you can do the whole thing in O(n) after you pay the cost of preprocessing to build the automaton, with a quite tiny constant for small patterns.

Grep is not that naive, afaik. https://news.ycombinator.com/item?id=2393587

Is this still the case if you set LANG=C on the command, so that you're lowercasing ASCII bytes instead of Unicode?

I don't think they're benchmarks though. I think the great part about this piece is that it gives people more intuition about computer speeds in specific use cases to identify bottlenecks better. If you have a complex operation like serving a web page, and you measure each part of the process, this page gives you a feel for what the ideal cases of file IO, memory access, computation, serialization and network access are so you can sort of tell what to fix a lot faster. Essentially a broader version of Numbers Every Computer Programmer Should Know.

pypy fixes python's slow class instantiation. The HTTPRequest request parsing is probably much faster on pypy because it creates a new class within the loop.

one of my fav performance bugs: https://bugzilla.gnome.org/show_bug.cgi?id=172099

Reported: 2005-03-30, unpatched to this day, because parsing opened files on the fly recursively with O(2^n​) complexity is enough.

Well there's always flags to prevent compiler optimizations, or maybe the example was purposefully presented in readable C, not whatever hack you'd need to do to bypass optimization. Inline assembly isn't exactly C anymore.

But yeah, I was surprised by the number of operations per second too. I was thinking it had to be over a billion.

> you can probably have a working board for about five bucks that'll run rings around an Apple II or a vintage PC.

Hell, you can do even better than that. Assuming that CHIP (https://www.kickstarter.com/projects/1598272670/chip-the-wor...) delivers on it's Kickstarter, for $9 you get a 1 GHz CPU and 512 MB RAM. That's roughly on par with an average home PC from about 2002-2003.

If you bump your budget up to $40, you get a Raspberry Pi 2 with a quad core 1 GHz chip and 1 GB of RAM. Now we're talking parity with an typical home PCs from 10 years ago, or less.

You'll be happy to know that computers can go even faster, and it doesn't need anywhere near 3.5 (= 115e6/33e6) bytes per number: you can use a single bit for each one (3.9 MiB), or only store numbers that aren't obviously composite (e.g. only odd numbers gives half that, and using a 30-wheel gives 1.0 MiB).

In any case, you can do a lot better than merely 33 million: e.g. http://primesieve.org/ uses some seriously optimised code and parallelism to count the primes below some number between 10 billion and 100 billion in a streaming fashion (meaning very small memory use). For non-streaming/caching the results, I'm not sure how primesieve does, but my own primal[0] (which is heavily inspired by primesieve) can find the primes below 5 billion and store everything in memory in 1 second using ~170 MiB of RAM on my laptop (and it doesn't support any parallelism, at the moment), and the primes below 500 million in ~0.75 seconds on a Nexus 5, and ~1 second on a Nexus S (although both devices give very inconsistent timings).

[0]: https://github.com/huonw/primal

I guessed it was one, and answered a couple orders of magnitude off. I'm going to give myself partial credit, since I actually thought about the work factor before answering. But not enough to go read the docs, unlike you!

There's a number of benchmarks at [1]. It would be nice if somebody would compile+run them on an AltJS environment and publish the result for different browsers.

[1] http://benchmarksgame.alioth.debian.org/

I do not think that it is that useful. While i have seen people greatly overestimate computing complexity of simple tasks like comparing numbers, i do not believe knowing how long such instructions take gives you much more. Especially interpreted JIT compiled code is not exactly predictable in performance and optimizing single instructions not very useful. However it is a fun game and does indeed give a better rough assessment for performance in various implementations. But if you really like to optimize your code these are just some hints. You will have to run benchmarks for every use case.

A high-level look at the relative speeds of common JS operations and patterns would be very useful. I often wonder if I'm wasting time thinking about how to optimize something when it won't make any practical difference.

Silly mortals and their non-N-body problems!

With Python this is usually a version mismatch - 2.x and 3.x are subtly incompatible.

That's probably a bad idea. This falls into the realms of pointless trivia and needlessly-specific experience in a narrow domain. If you're actually worried about optimization, these aren't the questions you would ask anyway.