&& operator chains (and ||, possibly) generates unoptimizable LLVM IR | Hacker News Reader

&& operator chains (and ||, possibly) generates unoptimizable LLVM IR

github.com

228 pointsest31100 comments

This stuff is fun. In the TXR Lisp compiler was running into pattern matching code refusing to optimize nicely. My test case was this Ackermann:

  (defun ack (:match)
    ((0 @n) (+ n 1))
    ((@m 0) (ack (- m 1) 1))
    ((@m @n) (ack (- m 1) (ack m (- n 1)))))

I then added an "early peephole" pass which works on the input code before it has been divided into basic blocks:

http://www.kylheku.com/cgit/txr/tree/share/txr/stdlib/optimi... (function at line 588)

There, I added recognition for a certain four-instruction kernel occurring out of pattern matching code, and rewrote it to three instructions.

Poof; I was left rubbing my eyes as out popped an instruction sequence identical to the one for this non-pattern-matching Ackermann, modulo some register names:

  (defun ack (m n)
    (cond
      ((eq m 0) (+ n 1))
      ((eq n 0) (ack (- m 1) 1))
      (t (ack (- m 1) (ack m (- n 1))))))

That early reduction kind of removed a logjam, allowing other optimizations to proceed.

The pattern matching code generates a flag which is set true upon matching, so that subsequent cases are skipped. This flag completely disappears.

The recognized instruction pattern spans (what would be) two basic blocks because it includes a jmp instruction. The rewritten instruction sequence eliminates one conditional branch, which gets rid of a basic block division, which is part of why it works.

Anyway, a big picture lesson here is that a big reason for the existence of pattern matching is so that so you have a tool that can be applied to optimize pattern matching.

This is more related to the story than meets the eye. early-peephole matches a pattern that will span two basic blocks if it is not replaced.

The rewrite removes a conditional which tests a temporary flag; that test ends up in a basic block by itself:

  (defun early-peephole (code)
    (rewrite-case insns code
      (((mov (t @t1) (d @d1))
        (jmp @lab2)
        @(symbolp @lab1)
        (mov (t @t1) (t 0))

A label is matched (backreferencing the earlier (jmp @lab2), so we know this item must be a label) followed by an ifq:

        @lab2
        (ifq (t @t1) (t 0) @lab3)

and so the above label and ifq form a basic block which does nothing more than tests a register to go somewhere else: the following code or some lab3 elsewhere.

This is very reminiscent to the numerous temporary-testing basic blocks shown the messy "unoptimizable" flow graph in the submission.

        . @rest)
      ^((mov (t ,t1) (d ,d1))
        (jmp ,lab3)
        ,lab1
        (mov (t ,t1) (t 0))
        ,lab2
        ,*rest))

In the rewritten pattern, that ifq is gone.

If we look at the diagram in the submission: some things are striking. For instance, have a basic block 14:

   bb14: fill temp5 with false

And another one:

   bb13: fill temp5 with true

Both of these jumps unconditionally to

   bb16: check temp5

The situation tested by my pattern above is exactly this sort of thing.

E.g. if we look at the pattern matching ack without optimization, we can find them:

  2> (disassemble (let ((*opt-level* 0)) (compile 'ack)))
  data:
      0: ack
      1: 0
      2: 1
      3: t

Note 3: t means that in this VM description, register d3 holds t, the canonical Boolean true symbol:

      [ snip ]


     31: 2C060403 movsr t6 d3
     32: 34000022 jmp 34
     33: 2C060000 movsr t6 nil
     34: 10000006 end t6
     35: 3C000036 ifq t6 nil 54

Here is an instance of the pattern. Except for the "end t6" has to do with closing a "frame ..." instruction earlier. We don't see this "end t6" in the pattern, and so it would cause a mismatch; but this instruction is removed by frame elimination, which is earlier in the compiler and happens at a lower optimization level.

So then, what do we have here?

    basic block A: "fill temp6 with true, jump to C"

     31: 2C060403 movsr t6 d3
     32: 34000022 jmp 34

    basic block B: "fill temp6 with false, fall to C"

     33: 2C060000 movsr t6 nil

    basic block C: "check temp6"

     34: 10000006 end t6
     35: 3C000036 ifq t6 nil 54

That very similar that aforementioned situation in that basic block diagram!

In my case, what breaks the logjam is that we get rid of the check, because the pattern knows that the check is only being reached from those two sources.

And then, register t6 succumbs to dead register removal. A subsequent data flow analysis, done after flow control optimizations (jump threading) discovers that these definitions of the t6 value have no next use.

I think that thanks to work done since then, that early-peephole thing could be moved into jump threading. There could be jump threading patterns which infer that the target of a jump is a conditional instruction which depends on the value of a register which is set to a true/false value prior to the jump, and then adjust the original jump. It's more general, but more annoying to code because of pattern matching across multiple discontinuous basic blocks.

Is the code up anywhere? I’d be interested in studying a lisp compiler.

Yes:

http://www.kylheku.com/cgit/txr/tree/share/txr/stdlib/

Files compiler.tl, optimize.tl. asm.tl.

Main entry points into it are the functions compile and compile-file, as well as compile-toplevel which is public and used by those two.

There's some interesting syntax in those files that I haven't seen before. What's ^((,me.(get-sidx 'exception-subtype-p) mean? Specifically, dot followed by open paren. Is it a method call?

In my own lisp, I settled on a convention like (foo (.bar 1) (.baz 2)) but that might be an interesting alternative.

^ seems like quasiquote. Is it?

Yes, ^ is quasiquote. The only other weird thing is that ,* is splicing, and not ,@. With that mapping, your quasiquote skills transfer.

` cannot be quasiquote because it's used for quasistrings. @ is also a notational prefix with a number of uses.

When an earmuffed special variable has to be inserted inserted without splicing, it is written , *variable* with extra whitespace. ,**variable* inserts *variable* with splicing.

   obj.(fun arg ...)

is indeed a method call.

Within certain constraints, a.b.c.d stands for (qref a b c d). The elements can be symbols or compound expressions. obj.(fun arg) is (qref obj (fun arg)). The qref macro transforms that into the method calls and slot accesses. From time to time qref appears in backquote templates. The syntax ^(lorem ipsum ,var.(abc)) calls the abc method on var and inserts the return value into the template, which is wrong if what we want is for just the value of var to be inserted into the template, in order to generate a method call on whatever syntax var stands for. That can only be directly written as ^(lorem ipsum (qref ,foo (abc))).

  ^(... ,me.(get-sidx 'exception-subtype-p) ...)

means that we call the get-sidx method on the me object with that exception-subtype-p symbol as its argument. The return value of that method calls is inserted.

get-sidx builds a table of symbols which have integer indices, referenced from the virtual machine code. The catch needs to be able to call exception-subtype-p at run-time, so it nails down an integer index for it and generates a gcall instruction referencing it.

The symbol index looks like a vector in the compiled VM description; the VM turns it into an array which provides efficient access to global bindings: the symbol is looked up the first time the index position is accessed, and the binding is forwarded into the table for faster access next time.

The objection I have about me.(get-sidx 'foo) as opposed to me (.get-sidx 'foo) is that get-sidx will be expanded by symbol macros, which you almost never want. Ditto for me.sidx: sidx will be substituted by any symbol macro named sidx, whereas me .sidx won't. (me.sidx can be expanded into me .sidx automatically, but most lisps, including my own, tend to expand it to something like ((idx me sidx)) which turned out to be a mistake, since sidx ends up expanded in later macroexpansion passes.)

It's not a theoretical objection, FWIW. I've been bitten by this in practice, and wince each time it comes up.

Cool lisp. What's it called?

This is TXR Lisp.

I assure you that me.(get-sidx 'foo) is (qref me (get-sidx 'foo)), where (get-sidx 'foo) is not an ordinary function call, but a macro argument that is never treated as a form to be expanded and evaluated:

From the repl:

  1> (expand 'me.(get-foo 'bar))
  (call (slot me 'get-foo)
        me 'bar)

So as you can see get-foo turns into a quoted symbol used for a slot lookup. The qref macro receives (get-foo 'bar) unevaluated and does this simple transformation.

Don't let the repetition of me fool you; a gensym will be used when it matters:

  2> (expand '(car x).(get-foo 'bar))
  (let ((#:g0012 (car x)))
    (call (slot #:g0012 'get-foo)
          #:g0012 'bar))

Likewise

  3> (expand 'me.sidx)
  (slot me 'sidx)

Again, by the me.sidx -> (qref me sidx) -> (slot me 'sidx) route. me.sidx is read syntax; nothing sees that but the parser. You can just pretend it doesn't exist and read it as (qref me sidx), just like you can read 'foo as (quote foo).

In any expression where some symbol sidx is susceptible for being treated as a symbol macro, that (almost) implies it is being treated as an evaluated form, which would be the bad thing. If we have some (. obj sidx) where sidx could be expanded, it means that sidx is being evaluated as a variable, which is the bigger problem. If the . operator can prevent that evaluation, surely it can prevent the expansion.

The reason for "(almost)" is that macros can do anything. We can have a (mac foo) which symbol-macro-expands foo, but doesn't evaluate it.

I have an operator like that in TXR Lisp called equot: expand macro, but suppress evaluation ("expand-quote").

  1> (symacrolet ((a (+ 2 2)))
       a)
  4
  2> (symacrolet ((a (+ 2 2)))
       (equot a))
  (+ 2 2)

But me.sidx doesn't do anything like equot on sidx; just regular quote!

I have my own Lisp compiler as well if you are interested: https://bernsteinbear.com/blog/lisp/

Bonus points for (a) being (essentially) a literate program, and (b) being in C. Thanks!

Plus, it has a nice exercise for he reader:

Comment above Compile_let:

  // This is let, not let*. Therefore we keep track of two environments -- the
  // parent environment, for evaluating the bindings, and the body environment,
  // which will have all of the bindings in addition to the parent. This makes
  // programs like (let ((a 1) (b a)) b) fail

But, let and let* can easily be handled by the same logic, with just some conditional adjustments. In some ways it is easier than interpreting both let and let* with the same interpreter code.

Here is why: in a compiler, we can fold the let* bindings into a single environment. The abstract model is that each new binding is a new environment which sees the previous bindings. But in fact, that doesn't have to be the reality.

To compile let and let* with the same logic all we have to do is:

- create an empty environment for the variables

- add variables to that environment as we iterate over the init expressions: after treating each init expression, we then add its corresponding variable to the environment

Then the difference between let and let* is that:

- for let, compile each init expression in the original, incoming environment

- for let*, compile each init expression in the new environment.

The extend-as-you-go logic takes care of the scoping.

(Note that because let* usefully allows duplicate variables, you have to make it possible to add the same variable to the env more than once, and be sure that the lookup finds the most recent one.)

The down-side of the flat environment is that a lexical closure captured by the init form of an earlier variable will capture later variables. The closure's body cannot see those variables, but the garbage collector probably can.

    (let* ((light-object (cons 1 2))
           (fun (lambda () light-object))
           (expensive-object (allocate-lots)))
       ...)

If fun escapes, the environment object it references will hold a reference onto expensive-object, even though the lambda only refers to light-object, unless the implementation goes out of its way to fix this. Whereas, if multiple cascaded environments were used (or a naive alist) then that entire part of the environment that holds expensive-object would become garbage, if the only object that is reachable is that lambda.

Interesting investigation. But it sounds like the discssion revealed that this may be "just" a regression.

A regression when the LLVM backend updated from 8 to 9. So arguably a regression in LLVM!

Reminds me of a Zig stream I watched recently where Andy was updating the compiler backend to use an LLVM 12 release candidate. He ended up filling a couple bug reports with LLVM, using Zig compiler issues to generate reduced case LLVM IR examples that compiled differently with LLVM 11. He mentioned that he had to try this upgrade with the release candidate because right now they're LLVM regressions, but once 12 ships they become Zig regressions!

Zig does a really good job at finding bugs in LLVM every time the LLVM version is updated.

And Zig developers also do a very good job at reporting and fixing them. Which is why contributing to Zig also means contributing to LLVM.

Language developers finding bugs in LLVM is ok, but if there are already several instances of regressions in LLVM found by developers of (maybe "exotic") languages, then the reaction should be less "hey, well done, you found a bug" and more "how can we avoid such regressions in the future?"...

I imagine Clang and GCC have growing test-suites to defend against regressions. I wonder if they share them.

They both do, yes.

LLVM's [1] includes (some of?) gcc's, the reverse may also be true.

LLVM also has extensive unit tests that can exercise codegen and backend functionality.

[1] https://github.com/llvm/llvm-test-suite

Clang writes a subset of LLVM IR, and other types of IR are supported but very poorly optimized/tested/etc. The LLVM docs say you should generate what Clang generates to avoid regressions.

Separate the backend into functional subcomponents and fuzz them as much as possible?

It's what cranelift does.

Fuzzing is great for finding crashes and other catastrophic misbehavior. But suboptimal codegen like this would be difficult to reveal with fuzzing.

You could do it with a reference compiler (this has in fact been done before) but finding suboptimal codegen like this case would still be kinda tricky.

Based on other comments this seems to be a regression in LLVM. LLVM does have a reference compiler for regressions: It's the previous version of LLVM. (Not saying that that makes this trivial.)

But if this next release makes codegen improvements, those would all show up as differences. Separating the improvements from the regressions is difficult enough -- fuzzing doesn't really help here, it makes it much harder to determine what code should have been generated.

Generally, codegen issues are exposed by benchmarks or someone who is curious enough to examine and analyze the code generated from multiple compilers or multiple releases. The latter is much rarer. But as these happen, the bar keeps getting raised and we do grow the test suite.

For those reading now it’s not LLVM’s fault. It was a rust change that disabled it

I think this should count as a bug in Rustc.

If your compiler generates IR that's just awful enough that it's a coin toss whether the backend can optimize it, whereas it can optimize the "naive" IR you'd write by hand, it's a compiler problem, not a backend problem.

This is a phenomenal bug report.

I reserve a different term for bugs where the backend is actually wrong (e.g. emitting the wrong SIMD instruction was a fun one)!

Wow! What a write-up! I appreciate the liberal use of the summary/details tags. Really helps things stay readable.

Curious: is there a Markdown exporter that supports some syntax for this?

The details tag is just html that you can fall back to in markdown: https://developer.mozilla.org/de/docs/Web/HTML/Element/detai...

The main contribution by Github's markdown renderer is that it doesn't remove that tag.

But how do they put code tags in there? Last time I checked, Markdown doesn't allow Markdown inside HTML tags. Or do they just wrap it in <code><pre> and the syntax highlighter picks it up later in the chain?

Markdown inside HTML tags is fine as long as there's a paragraph break (two newlines) between the Markdown and the HTML tag. So it usually goes like:

    <detail>
    <summary>...</summary>

    Some `markdown`
    
    </detail>

I would imagine this sort of behavior is in commonmark, but I am too lazy to check.

I’m really impressed by the diagrams (and they’re vertical so they’re mobile-friendly too!), how were they generated?

I think they used Graphviz: https://graphviz.org/

Meta: odd typo in the title, "-|" should just be "||".

I copy pasted the title from Github. I think the mismatch is caused by hn's headline mangling? I can't edit it any more, but an admin like dang could maybe correct it.

Whenever someone repeats the tired meme of "the compiler generates better assembly code than a human", I think of cases like this one (or the many occasions where I inspected compiler-generated assembly and it was atrocious). The rule of thumb should be "the compiler generates better assembly code than a human could within the time it takes to write high-level code and press 'Build project'", but that's not as catchy.

Your rule is much weaker to the point of being useless -- you are claiming that if it took me 1 hour to write high-level code, the compiler will generate better assembly code than what I can write in 1 hour.

This is obviously true in most cases -- if it takes me an hour to write task X in high-level language, it will probably take me a day or more to write it in assembly. After spending only one hour on assembly version the program will be nowhere close to completion, and likely completely non-functional. So whatever compiler produces will be much better -- it will at least work.

I can believe that at some moment hand-rolled assembly will "beat" generated code (I am pretty sure that if someone spends a month optimizing assembly listing it will beat the best Rust program written in a hour), but I am not sure what the ratio is. I am sure it is not 1:1, and I suspect the ratio gets worse as program grows and there are more parts that one needs to keep track of.

> Your rule is much weaker to the point of being useless

Yes, it's much weaker, but it's not useless. The vast majority of code in a program does not need to be as fast as it could possibly be [1], so there's no point in manually implementing everything in assembly. That doesn't mean that it never makes sense to manually implement something in assembly for performance reasons.

[1] And for many programs, all the code does not need to be as fast as it could possibly be.

Right, but I called the rules "useless" because it does not tell you any actionable info -- and this is normally expected of rules.

Here is a fixed version (note I added "times 20" at the end):

"the compiler generates better assembly code than a human could within the time it takes to write high-level code times 20"

See -- now I can make predictions. Let's say you have a function that you want to make fast, and you want to optimize it in assembly. The function took 30 minutes to write and debug. Do you have 10 hours to spare? Try assembly. No time? Leave as-is.

Much more useful this way, no?

Now it's more accurate, but even less catchy ;-)

How about: "don't blindly assume that the compiler is smarter than you"

That sounds like a special case of "nothing is perfect" and even less actionable. Especially with how vague "blindly" is.

Maybe "the compiler generates better assembly code per second than a human could"?

There may be some specific parts of assembly generation where compilers are still outperformed by men, just as there are still specific parts of chess where men still outperform machines, but overall machines beat human programmers.

The argument of allowing a human programmer to spend vastly more time is hardly fair, one could also probably write a hyper-optimizing compiler that takes far longer to compile to level the playing field, and that compiler will still triumph over men in how optimized the end result is.

The only real advantage overal to human programmers here is the flexibility. A man can easily be told “spend four months on optimizing this”, a machine would have to be substantially rewritten to fully take advantage of being given extra time.

> the tired meme of "the compiler generates better assembly code than a human"

This seems incomplete. Which human? The average software developer? That doesn't say much. Few developers have the necessary expertise. Expert assembly programmers are often able to outperform compilers.

Better wouldn’t necessarily be the right qualifier, but faster, typically more repeatable, and greatly more economical with scale/workload would certainly fit as better from different vantages.

With high novelty? Probably not until machine learning and compilers are deeply entangled.

That very much depends on the code. Program synthesis is an active area of research and the programs found would often be very difficult for a human to figure out. Of course a sufficiently determined human can always do whatever these programs do, but I do think it is unfair to give an unlimited amount of time to human optimizers.

That is one well written Github issue!

Rust is already very close to C and C++ performance, and getting to equivalence or possibly better will mean a long slog through these kinds of edge cases. This was already done in C and C++ compiler toolchains, which is why these languages are so fast, and it's a bit of a feedback loop because their popularity and centrality to systems development is what led to the work being done in the first place.

Rust + LLVM will already handily beat a naive bare bones C or C++ compiler that isn't loaded with sophisticated optimization tricks.

If the work is done I expect Rust to be faster than C and C++ for the same reason that C++ can sometimes be faster than C: a more advanced type system can allow better optimization in many cases.

In C++ for example you can do algorithms with templates, while in C you generally end up with function pointers all over the place. See qsort() vs. std::sort<> for one common case.

With Rust you can do all this generic stuff plus leverage better safety to do more strict aliasing optimizations, less unnecessary memory fencing, and so on.

I think Ada/GNAT performs roughly on par with C and C++, so we have reason to be optimistic. It's not just the dominant languages that end up with mature optimizing compilers.

Having been there when C compilers used to generate lousy code and we had legends like Mike Abrash teaching an whole industry how to write high performance Assembly, it always feels ironic that there is this myth about C and C++ being blazing fast since they were created.

C was fast.

On a PDP-11.

C++ was always taken as big, bloated and unusable compared to C.

Only by C developers that still believe C generates faster code than C++, while their beloved C compiler has been rewritten in C++.

gcc is not my daily compiler.

I found another tinyc user.

That's not what I believe. C generates the code I expect, rather than the insanely bloated crap that comes out of C++. C++ obfuscates what is actually going on. Oh, and then there are all the problems with the C++ standard library. It's unusable in high performance embedded style code due to many aspects of the standard library being unconstrained - things like memory allocations and unspecified complexity. In some cases we were able to use boost, but in many cases we had to reimplement many algorithms where the implementation performed allocations at init time, and no allocs at runtime. But oh how does an event driven, multithreaded and hot running messaging system go fast...

Then you expect very little from your compilers, unless you are just targeting 8 and 16 bit CPUs on the embedded market.

This is on modern x86-64 utilizing many cores and processing millions of messages per second. Efficient, well thought out architectures work well on small systems as well as large.

I bet that at -O3 it definitely does not generate the code you expect, and the same C code when compiled with a C++ compiler will generate even better machine code.

You would be wrong. I've spent plenty of time looking at code produced by the compiler from both C and C++ while working through performance profiling. Yes -O3 code can result in non-trivial transformations, but it's still easy enough to understand when one has been doing so for decades.

In my experience, it isn't the compiler that is the bottleneck to performance. Code can always be tweaked to get the compiler to produce output that does what is intended. My complaints about C++ are more due to the complete unsuitability of the standard library to embedded type applications caused us to spend plenty of time reimplementing algorithms that were already provided as part of normal C++ infrastructure. Having the right infrastructure to build a high performance application is far more than having an optimizing compiler. Libraries matter, kernel APIs matter, taking advantage of the right hardware configurations matters. The compiler is actually a fairly small part of the overall performance picture - doing things like pinning threads and avoiding cache line bounces between threads/CPUs buys orders of magnitude more performance than changing the compiler.

> If the work is done I expect Rust to be faster than C and C++ for the same reason that C++ can sometimes be faster than C: a more advanced type system can allow better optimization in many cases.

Every now and then I check in on whether LLVM can deal with rustc spamming "noalias" on all references. You can find the latest change in [1]. While in theory this unlocks _a ton_ of optimizations, noalias is used very rarely in C/C++ code so these compiler passes are not exercised a lot by existing LLVM tests and/or not realized in full.

[1] https://github.com/rust-lang/rust/pull/82834

This is why I think Rust could eventually be faster than C and C++ for a lot of things. The work has to be done though. You're right that noalias enabled optimizations are neglected because you can rarely use them in C code.

On the Rust side I think the language needs some way to annotate if's as likely/unlikely. This doesn't matter in most cases but can occasionally matter a lot in tight high performance code. It can allow the compiler to emit code that is structured so as to cause the branch predictor to usually be right, which can have a large impact.

That's tracked by https://github.com/rust-lang/rust/issues/26179 by the way.

Why not just use PGO (Profile Guided Optimizations)?

Sadly, PGO does not work with cross-language LTO, because of conflict of LLVM module name ('Profile').

A more advanced type system may also drive you into a corner where you make bad decisions wrt performance.

Can you give an example?

Not amelius, but one case that happened to me is that Rust requires wrapping a `T` in a `RefCell` if two closures use it as `&mut T`. This happens even if you the caller know that the closures are invoked from a single thread and do not invoke each other, and thus only one `&mut T` will be in effect at any time. This is because closures are effectively structs with the captures as fields, so both struct values (and thus both `&mut` borrows) exist at the same time even though their respective fields are not used at the same time.

Not only do you have to use `RefCell`, but you now also have panicking code when the `RefCell` borrow "fails", even though you know it can't. rustc is also not smart enough to notice the exclusivity at compile-time and elid away the RefCell borrow flag test and the panic branch.

    fn foo(mut cb1: impl FnMut(), mut cb2: impl FnMut()) {
        for _ in 0..10 {
            cb1();
            cb2();
        }
    }

    let mut x = String::new();
    foo(|| x.push_str("cb1,"), || x.push_str("cb2,"));

https://play.rust-lang.org/?version=stable&mode=debug&editio...

Fixed using `RefCell`: https://play.rust-lang.org/?version=stable&mode=release&edit... . Inspect the ASM and trace the use of the string constant "already borrowed"; you'll see it being used for the borrow flag test because it wasn't elided.

The equivalent non-Rust program could use String pointers for the two closures. I'm not sure whether they could be noalias or not, but at the very least they wouldn't need to generate any panicking code.

The "used from a single thread" aspect is a red herring: RefCell can only be used from a single thread anyway, and the compiler enforces this statically.

The "state" value in a RefCell is overhead, although it's fairly minor given that it doesn't need any synchronization to access. The extra panic branches are probably the largest overhead.

That said, these overheads stem from Rust's safety guarantees rather than its strong type system: you can have a language with a strong type system that does not do these checks.

Furthermore, there are of course ways to avoid this overhead within safe Rust: if you can use the type system to prove that the cell cannot be borrowed at the same time, then you don't need to do the checks, and in that sense a strong type system can actually help avoid overheads that were introduced by being a safe language.

>That said, these overheads stem from Rust's safety guarantees rather than its strong type system: you can have a language with a strong type system that does not do these checks.

The difference in semantics between a `&mut T` and a `*mut T` is a type system one. `&mut T` requires that two do not exist at the same time, regardless of whether they are used at the same time or not; this is the contract of the type.

>Furthermore, there are of course ways to avoid this overhead within safe Rust: if you can use the type system to prove that the cell cannot be borrowed at the same time, then you don't need to do the checks, and in that sense a strong type system can actually help avoid overheads that were introduced by being a safe language.

Correct, which is why I made the effort of pointing out that rustc is not smart enough to do it, not that it's impossible to do it.

This may be splitting hairs a bit, because we all agree that this is a good example where using Rust in this straightforward manner leads to suboptimal performance. But I agree with the grand parent that this is mainly an issue with safety, not with the type system itself.

To show why, consider two alternative languages.

“Weak Rust”: an equally safe Rust with a weaker type system. It might not distinguish & and &mut, but it would still need those checks, because you might use those shared references to break a data structure invariant. It would have to detect such unsafe usage at runtime and raise the equivalent of Java’s ConcurrentModificationException.

“Unsafe Rust”: a less safe Rust with an equally strong type system. It wouldn’t need to do those checks. In fact, that’s basically C++.

Apart from rust not being smart enough to see what you're doing in this sort of situation, the access pattern you're trying to use would actually result in undefined behavior with &mut pointers (or I assume C restrict pointers) because of the aliasing guarantees. For example one optimization you could imagine the compiler actually doing would result in the following

   let mut x = String::new();
   
   let str1 = x; (store x in a local pointer, no one else is touching it because we have aliasing guarantees)
   let str2 = x; (store x in a local pointer, no one else is touching it because we have aliasing guarantees)
   
   for _ in 0.. 10 {
       str1.push_str("cb1,");
       str2.push_str("cb2,");
   }

   x = str1; (restore x to the original variable before our aliasing guarantee goes away)
   x = str2; (restore x to the original variable before our aliasing guarantee goes away)

And you just:

- Leaked str1

- Created an x that just says cb2 repeatedly instead of alternating between cb1 and cb2.

Obviously it's possible to fix this problem by having different guarantees on pointers (C's pointers, rust raw pointers), but it's not clear that the occasional overhead of some metadata tracking (refcell) isn't actually going to be more performant than the constant overhead of not having aliasing guarantees everywhere else. The most performant would be obviously having both, but as we've seen with C asking programmers to go around marking pointers as restrict is too much work for too little benefit.

>the access pattern you're trying to use would actually result in undefined behavior with &mut pointers (or I assume C restrict pointers) [...] Obviously it's possible to fix this problem by having different guarantees on pointers (C's pointers, rust raw pointers)

Yes, I said as much in the last paragraph.

>but it's not clear that the occasional overhead of some metadata tracking (refcell) isn't actually going to be more performant than the constant overhead of not having aliasing guarantees everywhere else.

Generating panicking code where it's not needed is bad in general. It adds unwinding (unless disabled), collects backtraces (unavoidable), and often pulls in the std::fmt machinery.

Yes, in general it's almost certainly true that non-aliasing pointers produce more benefits regardless. My comment was in the context of the very specific example it gave.

You could rewrite this as an iterator, which would avoid this problem and make the code much nicer.

The hypothetical `foo` is a third-party library function.

(The real code which I reduced to this example is https://github.com/Arnavion/k8s-openapi/blob/1fcfe4b34a1f4f1... , and the callee does happen to be another crate in my control. While this can't be reduced to something like an Iterator, it can be resolved by making the calle take a trait with two `&mut self` methods instead of taking two closures. That still requires changing the callee, of course.)

Just use `UnsafeCell` instead of `RefCell` [1]: It has zero overhead, but you have to be sure that there's really no simultaneous write/write or read/write access – just like using raw pointers in C or C++.

[1] https://doc.rust-lang.org/beta/std/cell/struct.UnsafeCell.ht...

Yes, I'm not averse to using `unsafe`, but one has to justify it on a case-by-case basis. Eg if you're doing this in a library, then keep in mind that some users are very adamant about using unsafe-free crates, so you may prefer to take the hit.

> then keep in mind that some users are very adamant about using unsafe-free crates

Couldn’t you just put the use of unsafe as a default and add a feature flag to force the safe (but slower) behavior. Then you get the best of both worlds: those who don’t care get performance for “free”, while those who care can force it when they want.

If anything you'd have to go the opposite way: use safe by default and add the option to turn off runtime checks like bounds checks on slice access. Because when you write safe code, you tell the compiler about the invariants of your code, while with unsafe code, you keep them in your mind yourself. They might not even translate to any safe Rust constructs at all. E.g. if you pass a pointer in C, what is the recipient of the pointer supposed to do with it? Is the memory content initialized? Who is responsible for deallocation? On the other hand, if the compiler is told invariants in terms of safe code, it's easy to avoid any runtime checks for them.

The users I was thinking of were more along the lines of people that run cargo-geiger etc, which just looks for "unsafe" in the source rather than anything dynamic based on selected features.

FWIW, another option is to use `std::cell::Cell`. That only allows replacing the value rather than borrowing it in place, so you would have to take the value out and put it back after you're done with it, which also results in unnecessary code generation. But there'd be no branch and no panic, so the impact should be less than RefCell. There's also no borrow flag to take up space (not that it really matters when this is a single value on the stack).

This is a good point. I always forget Cell can be used with non-Copy types too.

In c/c++, Marking every argument as const throughout a deep call chain to later find some edge case where you need to mutate one member far down the call stack, and where this would not have broken the top level contract of the function. Forcing you to do expensive copies instead.

Only so long as it be mandatory to obey.

Rust was specifically designed so that one can ignore all the restrictions if one be confined and tell the compiler “I know better, and I know it is safe.”.

Of course, if one be wrong in such confidence, u.b. lurks around the corner.

I suppose one big problem with Rust is that it's less specified than in C what is and isn't safe, so it's harder to be so confident.

I write compilers for fun (hobby) and I just transpile my IR data structure to C++ (I'm also experimenting with other targets like Haskell (similar to agda compiler) and Rust (work in progress)). My friends usually treat this approach "too amateur" and a "real language" should compile all the way to assembly. The reason I don't do that is if you transpile to a well-known language and then manage that compiler to go all the way to machine code, you get insane amount of optimizations for free. Like C++ compilers are so good, you rarely need think about low-level optimizations, as long as your program is asymptotically sound (i.e. you don't keep random inserting into an array) it gives pretty good performance regardless.

But then you miss all the fun with doing a nanopass compiler.

Maybe that's not entirely true though, you can still generate C/C++ in the end I guess.

Nim compiler also generates C, so I'd say you are in pretty good company there.

The Idris 2 compiler compiles to Chez Scheme; the Idris 1 compiler compiled to C, and the former actually produces significantly faster executables than the latter.

I sometimes see programmers have very strange, irrational purity misgivings a particular case was someone who did not wish to fork off `grep` for some task, but actually wrote his own implementation which broke input into lines and used and manually searched for substrings; — he was initially convinced that this would be faster, or was at least “cleaner” than simply forking off `grep`, and piping to it.

The reality is that modern `grep` implementations are so optimized that writing something that was as efficient would probably exceed the complexity of his entire project.

I have never understood why many programmers seem to have an irrational dislike against forking off POSIX utilities when they target such platforms, and piping through them, which are often far more optimized than anything they've ever written.

One reason is that disparate impls of POSIX utils don't behave as similarly as one would hope. It's easy to cross the line from using flags that are guaranteed to be available to using flags and behaviors only available on your machine.

An example of a mistake I've made like this is relying on `find` on linux to default to searching the pwd if it doesn't have a positional arg. Another is `sed -i` requires a file extension on MacOS, but doesn't on Linux. You can catch these with tests to some extent, but how many people bother running their CI on MacOS just to avoid breaking other devs?

That said, I did work on a production system which happily shelled out to `dig` for DNS lookups for several years; everybody who saw it had a visceral negative reaction, but it also worked well enough that there was no good reason to spend time replacing it.

> One reason is that disparate impls of POSIX utils don't behave as similarly as one would hope. It's easy to cross the line from using flags that are guaranteed to be available to using flags and behaviors only available on your machine.

Certainly not, it's as easy as looking up the specification for anything else; what is and isn't POSIX is well documented.

> everybody who saw it had a visceral negative reaction

This is a reaction rooted in emotion, not reason.

> This is a reaction rooted in emotion, not reason.

Yes, humans are driven by emotion, not reason. Even when there is superficial rationality, it's mostly post hoc rationalization, not actual rationality.

> I have never understood why many programmers seem to have an irrational dislike against forking off POSIX utilities when they target such platforms, and piping through them

Many programmers have trust issues with code, and find it easier to write and comprehend a minimally-functional, narrow-scope implementation of their own than to get to know a general purpose utility enough to trust it. Also, lots of programmers would rather spend time on the experiencia of hacking out their own version than hacking out an interface to someone else’s code.

(And, yes, there's a whole lot of illogic and inconsistency in how this tends to manifests. But I think that there are very understandable reasons why a tendency in this direction is not only associated with predisposition to become a programmer, but also, among programmers, with tendency to develop deeper and broader skill as a programmer.)

If your explanation would be the genesis, then the same distrust would manifest when calling library functions, which does not seem to be the case.

The distrust seems to manifest from forking off a separate process invoked by it's name, not relying on externally written software. — it seems to be a distrust of forks and pipes.

I think what in many cases also plays is a fundamental misunderstanding of performance bottlenecks, with many programmers believing that forking off a program and piping to it is measurably slow, while in reality if the target program be well optimized it will be faster than a single heap allocation.

> My friends usually treat this approach "too amateur" and a "real language" should compile all the way to assembly.

Get new friends—no, seriously. I used to think like them, and hung out with similar minds. It is gatekeeping[0] in a way that, if internalized, will blind you to opportunity.

A few things I missed because I scoffed at them for not being "pure enough":

1) The entire Javascript ecosystem

2) Bitcoin, circa 2009

3) Countless SV startup opportunities from 2001-2015

4) ML and data science (e.g. the terrible tooling)

The success I have found always came with an open mind to people's approaches, even if they were not the ones I settled on. It took me way too long to learn this, but I hope I can change a few younger people's paths.

[0] https://www.urbandictionary.com/define.php?term=Gatekeeping

Not to be like your judgmental friends, but if you're going to use the approach that gives optimizations for free, why not just target WebAssembly / LLVM IR / Cranelift IR?

On the other hand, why yes? The OP found something that works for them and seems to give them the level of optimization they want. Presumably they also know C++ well, but possibly none of the technologies you mention well enough to "just" target them.

Because I don't know web assembly, llvm ir. I'm pretty experienced in C++ and Haskell and they have "good enough" performance for my toy projects, so why complicate things?

Besides you can compile C++ to webAssembly and llvm ir. (don't know about cranelift).

I'm a little suspect of the "Language x is faster than language y" talk - it seems too prone to programmer culture to test scientifically, beyond being able to prove that you can or can't end up with the same machine code from multiple languages.

And you can almost always write a program that executes faster in a "slower" language than a "faster" one.

> Rust + LLVM will already handily beat a naive bare bones C or C++ compiler that isn't loaded with sophisticated optimization tricks.

Well - sure. LLVM is loaded with sophisticated optimization tricks. If one compiler does sophisticated optimization tricks and the other compiler doesn't do sophisticated optimization tricks, the compiler with the sophisticated optimization tricks will be faster.

Tangent: Ten years ago or so, LLVM/Clang was touted as being head and shoulders better than GCC, the big new thing, the bees knees. Some of how it was better was obvious to everyone- readable error messages are pretty awesome. (GCC stole that idea pretty quickly) But the big thing that the LLVM/Clang people talked about was how blazingly fast compilation was compared to GCC. It was normal to take 1/10th as long to compile something. The code didn't run as fast as the code generated by GCC, maybe 50% of the performance, but that was a problem that they would eventually fix with more sophisticated optimizers. But compilation was fast!

Here we are ten years later, and LLVM/Clang generates code that runs almost as quickly as GCC. What did it cost, besides ten years of incrementally making the optimizer more and more sophisticated, a little bit better one little bit at a time? Well, it takes just as long to compile something with LLVM's sophisticated optimizer as it does to compile something with GCC's sophisticated optimizer.

The point is, there's a lot of yak shaving that will have to go into making Rust+LLVM work together better. And LLVM is already well acquainted with the barber. The low hanging fruit you're looking for has already been plucked.

It’s been a a bit since I was a C++ developer, but didn’t LLVM spur GCC on to improve significantly? Maybe LLVM and GCC are pretty comparable today with respect to build time and performance, but I think the more interesting question is how LLVM compares to GCC ten years ago (or more precisely, where GCC would be today were it not for competition from LLVM). Personally I’m glad there are multiple options, and people seem to prefer using LLVM for a language backend more than GCC for whatever reason, so it seems like a significant contribution to our digital society.

Hard to say.

In some places it's clear: LLVM's error messages were head and shoulders better than GCC's, and it was immediately obvious to everyone that GCC had to improve. And they're much better now.

But I don't believe GCC has made substantial improvements to its compilation speed since LLVM was introduced. In fact I believe it to be the opposite; as a Gentoo user I generally notice that compilation performance gets worse with every major GCC update. It's only due to hardware upgrades that compilation gets shorter.

LLVM is inherently much more modular than GCC is, which makes it a very attractive target for a language backend. It's a very large amount of work to shoehorn a language into GCC. The modular nature of LLVM is quite useful for lots of purposes, it is the back end for the syntax highlighting and error highlighting module I use in emacs. clang-format is a godsend for me, and unlike most other autoformatters, it formats based on semantics instead of syntax, which is a huge boon. The static analyzer and llvm-mca are also closely tied to the modularity of LLVM. It's not immediately obvious how one might implement these features based on GCC.

So yes, I 100% agree that LLVM is a hugely beneficial contribution to our digital society, even though it's not my first choice for a compiler.

Reply on news.ycombinator.com