Hacker News Reader

Comment by mpweiher | Hacker News Reader

>The only proper way to reason about the cost of these

>abstractions is to inspect the generated machine code.

To me, that's a big problem with a lot of these Heldencompilers. They may generate really optimal machine code. Then again, they may not, and the difference between optimizations working well and not working well in runtime efficiency is so great (I've measured 1000x for Swift) that they might as well be completely different languages.

For reference, 1000x means that 1 second turns into 16 minutes, and having that type of difference in something that's completely opaque is not a useful performance tool for me, because predictability is at least half the game in performance. So something like Knuth's transformation systems that turn optimization into a dialogue between programmer, compiler and instrumentation seems like a better idea[1].

[1] https://www.cs.sjsu.edu/~mak/CS185C/KnuthStructuredProgrammi...

View on HN

The problem is immature tooling. There is no feedback loop from the compiler's generated assembly back to the IDE. We finally have libclang which sorta does some stuff (I'm not entirely sure how far it can go) - but I'm honestly not seeing any work being done in this direction on the IDE level. After all these years of C++ development, why doesn't the IDE do something as simple as tell me if a function is being inline or not is beyond me (that's the tip of the iceberg in terms of what I want to know).

When I asked people at CppCon about it I just got some shrugs and was told "just go look at the assembly".

Another solution is profiling - but that's got a slow turn around, and it can be hard to narrow down problem areas.

For Lisp, SBCL is known for giving optimizations notes:

    ; note: forced to do GENERIC-+ (cost 10)
    ;       unable to do inline fixnum arithmetic (cost 2) because:
    ;       The first argument is a NUMBER, not a FIXNUM.
    ;       The second argument is a (INTEGER -19807040623954398379958599680
    ;                                 19807040623954398375663632385), not a FIXNUM.
    ;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES FIXNUM &REST T).
    ;       unable to do inline (signed-byte 64) arithmetic (cost 5) because:
    ;       The first argument is a NUMBER, not a (SIGNED-BYTE 64).
    ;       The second argument is a (INTEGER -19807040623954398379958599680
    ;                                 19807040623954398375663632385), not a (SIGNED-BYTE
    ;                                                                        64).
    ;       The result is a (VALUES NUMBER &OPTIONAL), not a (VALUES (SIGNED-BYTE 64)
    ;                                                                &REST T).
    ;       etc.

Note also that the approach is different there: integers are not modular, but you can still perform modular arithmetics if the range of values is adequate (http://www.sbcl.org/manual/#Modular-arithmetic).

Also SBCL has instruction level profiling.

Starting with VS 2015, Visual Studio does show execution time for each function call during a debugging session, while stepping.

Also switching between source view and Assembly alongside original source view is a key away (F12) for all Microsoft languages.

I'm not familiar with the term "Heldencompilers", but this describes compilers for all modern languages. :P C++ moreso than C, but even C can need cajoling to get it to conform to the specific structures that modern optimizers recognize (and that structure can even differ from compiler to compiler), especially for stuff like autovectorization.

Also note that Swift doesn't have the same performance goals that Rust does, preferring to trade some off in favor of ergonomics.

Swift's long term goals are to handle the exact same space as Rust, and that means the exact same performance/control goals. We're just starting higher up, and working our way down.

Rust is a lot more mature! RIP libgreen.

Can't wait to hear more about swift's opt in borrowing!

> this describes compilers for all modern languages

Yes, this is the current trend. And the results don't really justify the cost:

http://www.complang.tuwien.ac.at/kps2015/proceedings/KPS_201...

http://proebsting.cs.arizona.edu/law.html

Which is why I (roughly) agree with Daniel Bernstein's polemic "The Death of Optimizing Compilers" https://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf

(I frequently make code 10x to 1000x or more faster, and compilers' contributions to that total tend to be fairly minimal, though larger than zero and usually worthwhile. But not worth the current shenanigans and not worth having no idea how the code is going to turn out)

This also explains the shift in functional programming from languages with good reasoning about correctness (Haskell, but also SQL) to languages which additionally allow of good reasoning of performance (OCaml, Rust, etc.).

Moreover, this explains the NASA coding rule that all loops shall have an explicit upper bound.

The deeper issue seems to be: If you write code in a fashion that you can reason easily about its performance, then you might not get the optimal performance in all cases, but you establish an minimum level of performance.

So performance stability is more important than reaching optimal performance, because the latter may be easily destroyed accidentally in future versions of the software.

The only exception are well-defined tasks with stable input/output definitions (e.g. numeric primitives like matrix multiplication, fourier transformation, etc.) where the whole point of newer versions is performance improvements and nothing else.

I don't doubt that you make code 10x to 1000x faster, but could this be selection bias? I.e., you are focusing on the code that is slow and not the code that the compiler has already had substantial impact on?

You make code 10x 1000x faster by changing the algorithm or data layout that you use instead of just emitting different instructions for the same higher level code.

I don't think anyone who mattered ever argued than an optimizing compiler could or should attempt to turn bad algorithms into good ones.

It seems clear to me that at the present state of tech that Most developers are better at picking algorithms while compilers are better at picking which low level instructions. Clearly there are some exceptions, but things like gotoBLAS are clearly exceptions and not the rule.

No, but there can be this myth that certain things will be "optimised away" by the compiler, which is why you can get single digit or fractional speedups by just tuning the optimisation level.

I would much rather deal those "myths" than with the class of myths surrounding i++ being faster/lower than ++i or any of the non-sense around divisions.

At least people claiming something will be optimized away can readily be empirically tested in a way that most will listen to. I have no problem testing the / operator, but I have never had holders of such beliefs accept my results, but people claiming something like "virtual function calls can be optimized away" can handle it when I can demonstrated that inside a single binary they can be but at places like library boundaries they cannot.

> I would much rather deal those "myths" than with the class of myths surrounding i++ being faster/lower than ++i or any of the non-sense around divisions.

This is one area I dislike in C and C++ culture, there is this tendency to micro-optimize code like that, without even profiling it, or it causing an actual impact on the application being built.

So one ends up with endless bike shedding discussions about how to write code, instead of writing it.

That is a facet of programming culture. So many programmers in so many languages have their silly myths.

A few years some Ruby dev said that declaring methods without parenthesis would let the Ruby parser parse method declarations faster. The dev never even measured, he just thought the parens were ugly and a ton of gem devs removed parens claiming a speed up. It wasn't until a few years later that someone benchmarked it and determined that there either was no difference or that it cost just the tiniest amount to remove them.

Those are the things that the compiler can do to optimise your program by a few x. The sorts of optimisations that it can't do are things like picking a different algorithm, or reorganising the layout of a class so that it or multiples of it can fit within a cache line.

The solution is quite simple: you write using the abstractions and then profile; inspect the assembly for the hot parts of the code and tweak if needed.

I don't think I've saw someone look at the assembly of production code. Ever.

Well, maybe they should? Don't really see how else you're going to figure out why a line of code is as fast/slow as it is.

I can work out why stuff is going fast slow all the time without going near assembly, but I do higher lever web / db stuff. For the majority of poorly performing code it is blindingly obvious why.

Well, we do it, even if you don't see us. Tell your friends to do it too!

I do it at least once a week. Certainly C programmers do this, so inasmuch as rust is working to be a better C, once might expect Rust programmers to do it.

Swift does not have many free abstractions - what specifically are you referring to?

(Also Swift's pervasive atomic refcounting means that typical idiomatic code often has pretty atrocious performance.)

Keep in mind that Swift's emphasis on value types means that idiomatic code can have very little refcounting. Obviously that's not the case when you're doing iOS/macOS applications, since you're using the ObjC runtime.

Refcounting can also be dropped to non-atomic by the optimizer when it can prove that an object hasn't escaped.

Value types only avoid refcounts if they contain only primitive fields, which is not all that often in interesting code.

For example, count the number of atomic refcounts on each iteration of the following loop:

  // given
  struct S {
    var x: [Int]
    var y: [String]
    var z: NSView
  }

  var array: [S]

  for element in array {
    // maybe do something with the NSView
  }

That's three atomic increments and three decrements per loop iteration, as the array entry is copied into element because it can't prove the NSView operation wouldn't cause the array to zero somehow.

Given atomic memory operations are some of the most expensive things on a modern CPU, this is brutal, and a place where Rust-style memory ownership would be a big improvement.

I haven't seen the optimizer convert atomic refcounts to non-atomic refcounts - do you have an example situation where that happens?

(By the way, I think Swift is a delightful language, but the pervasive reference counting threw some cold water on my passion as it means today we're not really close to replacing C/C++/Rust for machine-level performance.)

That's exactly the point: the optimizer can drop atomicity. All these ref-counts may be optimized away. Or not. The opposite of predictability.

That is what profilers are for.

Even with pure Assembly, changing between CPU releases can have a big impact due to microcode changes or cache sizes.

Heldencompiler? Google is not turning up much.

Might be a typo of heisencompiler?

Heroic compiler?

Probably supposed to be Heisencompiler as a play on Heisenberg's uncertainty principal.

Is messing up branch prediction "Breaking Bad?"

No. See Heisenberg uncertainty principle.

From context, no, it seems to be "Heroic Compiler".

Nope. But I like it :-)

Helden is german for "hero"

"helden" is German for "heroes", concatenating words is fine in German grammar, I'm guessing he means compilers that try to be the hero and write their own code, rather than dumbly transform code into instructions.

"helden" is also Dutch for "heroes" and concatenation like that is also fine in Dutch.

Because, well, Dutch is kind of a glorified German dialect ;)

Just kidding, but German and Dutch share a close, common "ancestor" language, closer than any other Germanic languages. I used to have a professor at Uni who would always repeat, that a language is a dialect with an army, which kind of hits the spot.

My term for stuff like this is "software that suffers from overly-clever syndrome".

If you're saying 1000x for Swift, my immediate guess is you've got some pathological code that is triggering a massive number of copies of CoW structures (e.g. arrays, dictionaries, sets).

I am pretty sure it wasn't, and I wasn't trying to construct a case either. I haven't run into the CoW issues myself yet, though with that I am actually curious enough to try to construct a case.

Why are you pretty sure it wasn't? Swift is usually pretty efficient. Massive numbers of CoWs is by far the most likely culprit for any kind of huge slowdown. And without actually profiling it, it's hard to detect when CoWs are happening, you mostly just have to be aware of when you're writing code that keeps multiple references to a CoW data structure that you're mutating. It's very common to get single accidental CoW copies. Getting a large number of them tends to require slightly odder code. For example, saying something like `col.reduce([], { $0 + $1.elements })`, which is a normal idiom for Haskell, will introduce a CoW copy on the accumulator array at every single step. And this is true even if you modify it to something like `col.reduce([], { var ary = $0; ary.append(contentsOf: $1.elements); return ary })`, because behind the scenes the reduce method will still have a reference to the accumulator while the closure is executing.

How much speedup was that "1000x" actually though? Like 2x?

Reply on news.ycombinator.com