Hacker News Reader

Comment by hinkley | Hacker News Reader

At the end of this he says transistors are now doubling every twenty years(!?) and it reminded me of another law Patterson doesn’t include in his graph:

    Proebsting’s Law: improvements to compiler technology double the performance of typical programs every 18 years.

View on HN

The derivation of that law is very suspect:

http://proebsting.cs.arizona.edu/law.html

(go on, it's just a paragraph.)

The key issue that this ignores in my opinion, is that a compiler optimization will rarely make last year's program faster, but it will make next year's program faster. Why? Because if the compiler can't make an optimization, programmers will do it by hand, even if it makes the code worse in some way.

For instance, if your C compiler can't inline small functions, you would use a macro instead. When it finally starts learns to inline, your program won't get any faster, but the next version will be able to use functions in places where macros are a bad fit.

Pile up enough of these optimizations, and eventually it starts to feel as if you're coding in a higher-level language than before, even though the syntax that's accepted by the compiler never changed.

> programmers will do it by hand

Only if better performance is needed.

Thus, corrollary: compiler technology will double program performance every 18 years, but only if it doesn't matter.

Developers have a nasty habit of convincing themselves that things aren’t needed when they see them as too difficult. Even if the rest of the world thinks your code is too slow you can convince yourself it’s good enough.

And in a world where we rely more and more on libraries, my ability to improve on a piece of code is greatly curtailed. Sending in the compiler to help might be my best option.

That is an amusingly awesome claim.

I thought it was accepted that algorithm improvements have sped things up more than processor advances. I suspect there is a strong argument that memory sizes has been key, but processor speeds themselves haven't necessarily advanced at the same rate as the speed we complete problems.

That said, I tried quickly googling for this, but just came up with https://cstheory.stackexchange.com/questions/12905/speedup-f.... Looks like a good answer, but basically points out that it is complicated.

For my part, it is frustrating to see so many folks rediscovering things that used to just be too expensive to do and think they have rediscovered alchemy. I say this as someone that constantly thinks to have discovered a key method. :)

My experiences don’t jive with your thesis.

When people aren’t looking at a performance chart that is flat, they stop. No matter how loud the business is about the app being too slow people are too quick to announce that everything that can be done has been done.

Really in this situation there might be another order of magnitude hidden in there but it takes a special set of skills and a very special kind of perserverence to continue digging into a pile like that. A compiler has no such problem and I’m sure it could continue to shave off time for quite a while.

Correction: when people are looking at a chart that is flat.

> compiler optimization will rarely make last year's program faster, but it will make next year's program faster.

This doesn't follow. From your argument, last years and next years programs will run the same speed (about as fast as they can). It's just that next year's programs can be cleaner in some sense...

Which is interesting, because Dr. Proebsting's page also says his current interests include improving programmer productivity by removing syntactic baggage from statically typed languages.

Also plenty of optimizations happen at different layers than those effected by compilation flags. Constant folding, smarter register spilling, improved implementation strategies (e.g. virtual function tables). These are all techniques that still happen at -O0.

Doing by hand it is only worthwhile if the application does not meat the expected deadlines and only on spots validated through the use of a profiler.

I never cared about the C culture of speed before correctness, because type safety never impacted the expected use of my applications.

I care a lot about speed, but correctness is my foremost concern. I agree that I only look to optimize bottlenecks, so compiler technology would still optimize some of the less bottlenecky code from yesteryear.

Great!

Sadly not everyone does that.

I do agree there are domains where every ms and byte counts, they are however a small niche.

I think Proebsting's law [1] overlooks a number of points. Firstly, it is my understanding that compiler optimization made RISC feasible, and if so, then some of the hardware performance improvement over that period is attributable to compiler technology (at the very least, perhaps he should have compared today's optimizing compiler to a decades-old optimizing compiler, not today's compiler, optimized vs. unoptimized?) Secondly, he is extrapolating from at most two points (assuming his 'before' numbers are valid), and thirdly, with the (comparatively) low-hanging hardware fruit all gone, compiler optimizations have become relatively more useful (especially the work on optimizing concurrent and parallel software.)

Nevertheless, I tend to agree that programmer productivity is a worthy goal (and more specifically, those that improve productivity through making it easier to understand programs, so that programmers can more quickly produce programs that work properly.)

[1] See Marvy's comment for a link to the law: https://news.ycombinator.com/item?id=16751813

A compiler can only reduce the overhead relative to the hypothetical perfect program for a given task/source. That's asymptotic growth, not exponential.

Reply on news.ycombinator.com