Hacker News Reader

Comment by bjackman | Hacker News Reader

A potential lesson here (i.e. I am applying confirmation bias to retroactively view this article as justification for a strongly held opinion, lol):

Unless you are gonna benchmark something, for details like this you should pretty much always just trust the damn compiler and write the code in the most maintainable way.

This comes up in code review a LOT at my work:

- "you can write this simpler with XYZ"

- "but that will be slower because it's a copy/a function call/an indirect branch/a channel send/a shared memory access/some other combination of assumptions about what the compiler will generate and what is slow on a CPU"

I always ask them to either prove it or write the simple thing. If the code in question isn't hot enough to bother benchmarking it, the performance benefits probably aren't worth it _even if they exist_.

View on HN

Blog author here. I somewhat agree, somewhat disagree. This line makes me uneasy:

> I always ask them to either prove it or write the simple thing. If the code in question isn't hot enough to bother benchmarking it, the performance benefits probably aren't worth it _even if they exist_.

One of my philosophies is that death by a thousand cuts is fine, but death by ten thousand cuts isn’t. A team of 10 engineers can probably fix most of a thousand cuts in two or three months. But if you have ten thousand cuts you’re probably doomed. And those don’t show up cleanly in a flame graph.

Now for some context my background is video games. Which means the team knows they need to hit an aggressive performance bar. This isn’t true for many projects. shared_ptr is a canonical example of death by ten thousand cuts.

That said, I strongly agree with the principle of “just do the simple thing”. However I think it’s important to have “sane defaults”. A project can easily have a thousand or ten thousand papercuts that kill performance. But you can’t microbench every tiny decision. And microbenches are only a vague approximation of what actually matters.

I’m also wary of “the compiler will make it fast”. Because that’s true… until it’s not! Although these days you don’t have any choice but to lean heavily on the compiler and “trust but verify”.

No one wants a super complex solution if it’s not needed. However I am very amenable to “do a slightly more complex thing if you know it’s correct and we can never think about this ever again”. It’s much easier to do the fast thing upfront than for someone else to try and speed it up in two years when we’re doing a papercut pass.

> But if you have ten thousand cuts you’re probably doomed. And those don’t show up cleanly in a flame graph.

I am reminded of the lovely nanosecond/microsecond talk by Grace Hopper. If your code does a little bit of setup and then spends all of its time in a single hotspot, fine. But if your code is full of microsecond-suboptimal speed bumps, you can probably hide your hotspot altogether. And a flat-ish flame graph looks fine: nothing stands out as a problem!

It's valuable to do micro-benchmarks, not just to hone your optimization skills, but to learn optimal patterns in your language of choice. Then, when you're "in the zone" and laying down new code, you just do the optimal thing reflexively. Or, when you're reviewing or rewriting something, those micro-hotspots jump out and grab your attention.

There's a reason that ancient software running on ancient hardware is way more responsive & snappy than what we have today. Laziness.

> There's a reason that ancient software running on ancient hardware is way more responsive & snappy than what we have today. Laziness.

Laziness in terms of using entirely inappropriate algorithms, sure.

Laziness in not microbenchmarking minutia? It shouldn't be. There's a limit on how much that can hurt you. I would say much less than a factor of ten, but let's go with 10x just for argument's sake. If you have a CPU that's 500x faster, and use easy code that's 10x slower, you're doing just fine. This is not the problem with modern unresponsiveness.

When I rewrite python code in C, I often hit 1000x speedups, and sub-100x is rare. And that's line-for-line. When I fix an accidentally-quadratic issue, for example, I've seen speedups in the billions without even changing the language.

People have lionized Knuth's quote about premature optimization, and used that to ignore performance issues across the board. Since the early '00s, we have not seen a 500x improvement in CPU speed. It's less than 2x on frequency, and let's say 8x on core-count for most users (which doesn't help your single-core lazy programmer). In my experience, programmers will make projections based on a 500x-faster processors that will never arrive, because it's easier than honing their skills and keeping them sharp. And even if these magical THz-frequency chips arrive, if you have three layers of 10x slowdowns, you're back down to GHz.

This has been my experience too. I wrote a text crdt last year which improved automerge’s (then) 5 minute runtime. My code currently takes 6ms to do the same work.

Automerge’s design assumed this stuff would always be slow, so they had this whole frontend / backend code split so they could put the expensive operations on worker threads. Good optimizations in the right places make all that complexity unnecessary. The new automerge is shaping up to be simpler as well as faster.

> When I rewrite python code in C, I often hit 1000x speedups, and sub-100x is rare. And that's line-for-line. When I fix an accidentally-quadratic issue, for example, I've seen speedups in the billions without even changing the language.

And neither of those is a microbenchmark thing, which is kind of my point. I'm surprised language would hurt that much, but that's enough to break things on its own without any layering.

> Since the early '00s, we have not seen a 500x improvement in CPU speed. It's less than 2x on frequency, and let's say 8x on core-count for most users (which doesn't help your single-core lazy programmer).

I don't think people are talking about 2004 when they talk about the responsiveness of ancient software on ancient hardware. I interpret that as more like an Apple II. But instructions per clock have also gone up a lot since the pentium 4 days, and having more than one core in your CPU has a huge impact even for single-threaded programs.

> And neither of those is a microbenchmark thing, which is kind of my point. I'm surprised language would hurt that much...

The point I'm making here is that every line matters -- not just the hotspots. If you're suprised that language can have that much impact, perhaps it's time to learn a bit about performance issues that you're being dismissive of?

> I don't think people are talking about 2004 when they talk about the responsiveness of ancient software on ancient hardware.

No, I was responding to your mention of a 500x improvement in hardware. That pipedream ended in the early 00s, and people still talk like Moore's law will absolve their inattentive coding practice. And that felt fine in the decades we went from kHz to GHz, but it's unacceptable today.

> The point I'm making here is that every line matters -- not just the hotspots.

That depends on how it would have performed if you only transformed the hottest 10% into C.

But I was mainly responding to the idea that micro-optimizations are needed to keep general software snappy, and I don't think they are. If one language is that much faster, that's not micro-optimization.

> No, I was responding to your mention of a 500x improvement in hardware.

What do you mean "No"? I was talking about current hardware being 500x faster than 1988 hardware, which it is. If that's not what you meant by "ancient software on ancient hardware", fine, but that's what my 500x was talking about.

> people still talk like Moore's law will absolve their inattentive coding practice

I'm not trying to excuse inattentive coding. I'm trying to say certain kinds of attention are important and others aren't.

Python is easily 200x slower than C for equivalent code. The reason why Python is usable is because it uses a lot of C libraries that do the heavy lifting.

> And a flat-ish flame graph looks fine: nothing stands out as a problem!

If your program is still slow, that would also indicate that everything is a problem, ie. the ten thousand cuts. Start optimizing at some obvious spots and then see what happens.

Very little of your code is in hot loops. If the code that takes half a millisecond per frame could be twice as fast, but the hot loop is very optimized, then it doesn't really matter. And that's what I would think of by default for having many many cuts. Better to spend the optimization effort elsewhere.

> shared_ptr is a canonical example of death by ten thousand cuts

Why does that count as ten thousand cuts rather than one cut? That doesn't sound intractable to fix if you have months.

> Very little of your code is in hot loops.

Very few programs have "hot loops" as small, contained things. "hot loops" is at this point as much of a bad trope as "but premature optimizations!" is.

> Why does that count as ten thousand cuts rather than one cut? That doesn't sound intractable to fix if you have months.

Why spend months re-writing code you could have just written correctly the first time if a single person had spent an hour messing around with benchmarks to figure out the proper guidance?

> Very few programs have "hot loops" as small, contained things. "hot loops" is at this point as much of a bad trope as "but premature optimizations!" is.

If you say so. I can pretty easily and meaningfully divide my current program into "happens once per frame or less" and "happens hundreds or more times in a row".

> Why spend months re-writing code you could have just written correctly the first time if a single person had spent an hour messing around with benchmarks to figure out the proper guidance?

If they could have figured it out that easily I understand even less how this example works.

Let me put it this way: If a benchmark can change huge swaths of significantly performance important code, then it's "hot enough to bother" by orders of magnitude. I thought we were talking about microbenchmarks for individual cuts!

Haha "death by a thousand cuts" is exactly the phrase I encounter in these debates!

And actually I still disagree - e.g. I once took over a DMA management firmware and the TL told me "we are really trying to avoid DBATC so we take care to write every line efficiently". But the thing was that once you have a holistic understanding of the systems performance you tend to find _only a small fraction of the code ever affects the metrics you care about_!

E.g. in that case the CPU was so rarely the bottleneck that it really didn't matter, we could have rewritten half the code in Python (if we'd had the memory) without hurting latency or throughput.

Admittedly I can see how games or like JS engines might be a kinda special case here, where the OVERALL compute bandwidth begins to become a concern (almost like an HPC system) and maybe then every line really does count.

I generally agree, but it’s also not obvious to me in Rust (or in Go) whether passing by reference or by copy is more maintainable or clear. I guess what I want is some guidance on what I should do by default, which you sort of give with “do what is more maintainable”, but I can’t tell what that means in practice (I’ve been told to default to pass-by-reference in the past because most traits take &self and not self).

I don't think one of them is more clear or maintainable universally, but in a lot of contexts, there might be an obvious choice. As a trivial example (that isn't quite fair given that the topic is about structs), it will almost never be more clear or maintainable to pass a shared reference to an integer (although there may be cases where a mutable reference might make sense). I don't think there's much need for one to have precedence over the other by default; if anything, I see the discussion about performance tradeoffs not being worth fretting about in the absence of actual measurement to be an argument _against_ one of them being inherently preferable.

> I’ve been told to default to pass-by-reference in the past because most traits take &self and not self

This is only blanket advice for designing traits, because as the trait author you don't know what concrete type the downstream user is going to want to use, and taking `&self` in that circumstance is the choice that is friendliest to both Copy and non-Copy types.

If you're just writing a non-generic function and you do know what concrete types you're using, the flowchart is pretty simple:

1. If the type is not Copy, then pass by-ref if you just need to read the value, pass by-mutable-ref if you just need to mutate the value, and pass by-value if you want to consume the value.

2. If the type is Copy, then pass by-value, but if your type is really big or if benchmarking has determined that this is a critical code path then pass by-ref.

I would still consider myself a go novice, but I have been burned a number of times passing simple objects by reference and then that object gets mutated causing subtle bugs. Also, go is happy to blow your foot off if you take the reference of a loop variable. Although, there is a proposal to fix that.

Generally I find that less bugs get introduced when using copy instead of pass by reference, but I’m sure others have the opposite opinion.

I have had the same results. Passing by copy is simpler and less bug-prone and reduces the urge to "just set the value since I have a reference to the object" which is a well-paved road to significant pain.

And the objects have to get surprisingly large before passing by reference really makes a difference.

That kind of problem doesn't happen in Rust though, because if all you have is a reference then you cannot mutate anything. You can only mutate things if you have a mutable reference to them, which makes the caller jump through hoops to acquire one for you. They may have to have ownership of the thing, or use a Mutex or RWLock to acquire exclusive access to it, before they can call your method. Nothing can be mutated by accident, or in a surprising place.

This is about rust though and thats not really possible there (at least to my knowledge). You should get a compiler error if you attempt this.

I got very little experience in rust though, so there might be a way (I'm just not aware of) to circumvent this check

Indeed, you can use interior mutability to accomplish this:

https://doc.rust-lang.org/reference/interior-mutability.html

Usually (read: near universally) something that provides interior mutability will implement a runtime check to protect you (eg, a lock), but you could make a footgun if you were cavalier about it. And recent I misused a framework (I kept a value returned by the framework around that should've been dropped) without realizing it was holding locks internal to the framework, and ended up with a tricky to diagnose deadlock (though only tricky before I hadn't read the documentation as closely as I should've).

Ouch, I actually feel Ada was right in having the for-loop 'variable' be constant inside the body of the loop. If you want a modifiable loop variable, use a loop/while loop, but at least the first question in peer-review becomes 'why not a for-loop here'?

I don’t think that solves the problem because the loop itself still needs to mutate the variable. The issue often arises when people fork a new goroutine that closes over the loop variable inside the loop body—the loop often completes before any of the threads start running, so many/all of the threads read the loop variable at its max value (e.g., if we are forking 10 goroutines that just print their loop variable and quit, then there’s a good chance every goroutine will print 10 rather than printing 1-10).

Wouldn't any sane fork system capture the value of the loop variable instead of a reference to it? Why would anyone ever want to capture a reference to a loop variable that will be moved under one's feet? I'm sorry I'm struggling to understand what we're doing here and why.

No, the capturing is done by the closure and as far as I know capturing by reference is pretty standard for closures across languages. The forking system just runs the closure in a thread, agnostic to any capturing details.

Yeah, const semantics are one of many things I would have preferred to have in Go over generics. You can sort of emulate them with pass by copy, but (1) that has performance implications and (2) if your data contains a reference, that can still be mutated.

Yeah totally agree it's not always/usually obvious. But there are cases where there's a clear readibility/assumed-performance tradeoff and in those cases I say always prefer readibility (unless you benchmark).

> but that will be slower because it's a copy/a function call/an indirect branch/a channel send/a shared memory access

I really dislike these takes. I see engineers optimize these cases and then go ahead and make two separate SQL queries that could be one, ruining any false optimization gains they got by lord know how many times.

Yeah, you can loop over that 100 element list twice doing basic computation if you want, it's not going to make a difference for many engineering workloads, but could make a big difference in readability.

This x 10000 ! If I had a dime for every time I provided this exact feedback in code reviews…I find surprising that a lot of devs in tech industry are obsessed with pointless micro optimizations and they don’t care about writing maintainable, testable simple code. My final comment is always to not outsmart compilers/jvm because they tend to do a much better job that developers.

Please, don’t optimize unless you have reasons to do so and numbers backing that up.

This is true for application code. But, Rust is trying to sell itself as a systems language and an embedded language and a language you can write kernel modules in. Memory budget matters in these cases.

If memory budget matters, you have a memory budget. So you should be measuring and you can actually tell where you need improvements.

But in practice what we see overwhelmingly is that people want to do this stuff but they aren't measuring, because measuring is boring whereas making the code more complicated to show off how much you think you know about optimisation is easy. Knock it off.

Then knock off trying to use Rust as a systems language. Linear types make refactoring this nearly impossible if it does become and issue.

Works really nice for me, of course, I actually measure what I'm doing.

You edited your comment, so I guess I will too: Rust doesn't actually have Linear types. Linear types ("must use or compile error") would be tricky to provide, Aria blogged about it back in the day. So that's definitely going to be a problem with your refactoring.

It depends on your specialization, I guess. If you're making a website, a few microseconds here and there probably don't matter.

But in my field (Fintech), performance really does matter. Doing the simple, slow thing is just lazy and won't make it through review.

Great. Should be easy to prove then.

Yup, we have a good benchmarking suite to make sure changes don't cause regressions and that optimization changes actually work.

That said, I think asking a developer to write everything they do twice so that they can A/B test is overboard. You can come back and really aggressively optimize later, but I think the "default" should be the fast thing, rather than the slow & easy thing.

If "performance really does matter" (in Fintech) then a developer surely can write everything twice or more.

We could, yeah. I'm sure you're capable of re-writing everything you do twice. It's just a huge waste of time.

You get just as much benefit by assigning a performance refactor to an engineer when needed vs literally halving or worse the whole teams productivity.

Time to market can matter in fintech as much as anywhere else.

Edit: also optimizations often interact with each other and you can't always benchmark all possible combinations, so sometime you do have to rely on experience or reasoning from first principles to decide if an optimisation is worth it or not.

The idea is that the "default" is the easy thing, which is usually optimized by the compiler.

When you're writing in a language like Rust or C++, you know broadly what the compiler optimizes and what it probably won't. You do need to do the work yourself a lot of the time.

My advice is the opposite: if you want to make performance justifications for code, you need a benchmarking suite. I have them for a lot of my projects. (Rust’s criterion is a delight). A good benchmark suite is a subtle thing to write - you want real world testing data, and benchmarks for a range of scenarios. The benchmarks should be run often. For some changes, I‘ll rerun my benchmarks multiple times for a single commit. I benchmark the speed of a few operations, serialisation size, wasm bundle size and some other things.

Having real benchmarking data is eye opening. It was shocking to me how much the wasm bundle size increased when I added serialisation code. The time to serialise / deserialise a big chunk of data for me is 0.5ms - so fast that it’s not worth more microoptimizations. Lots of changes I think will make the code slower have no impact whatsoever on performance. And my instincts are so often wrong. About 50% of microoptimizations I try either have no effect or make the code slightly slower. And it’s quite common for changes that shouldn’t change performance at all to cause significant performance regressions for unexpected reasons.

I’ve also learned how important “short circuit” cases can be for performance. Adding a single early check for the trivial case in a function can sometimes improve end to end performance by 15-20%, which in already well tuned code is massive.

Performance work is really fun. But if you do performance tuning without measuring the results, you’re driving blindfolded. You’re as likely to make your code worse as you are to make it better. Add benchmarks.

One neat thing here is that the compiler is aware of which types are `Copy` and not internally mutable (not contianing an `UnsafeCell`). For these types, passing `&T` and `T` are equivalent, so the compiler could just choose the faster option.

Even if it's not smart enough to do that today, it could implement this optimization in the future. This could work even without inlining, since the Rust calling convention is unstable, and an optimization based on type size could be incorporated into it.

It would be more advisable to add this as a clippy hint, because `&T` and `T` are not always equivalent wrt. FFI.

Indeed, but the compiler is still capable of doing it on a case-by-case basis. Quite often the observed semantics are identical and it's easy for the backend to see that a pointer has been created only to be immediately dereferenced.

It would be nice if Rust could do this, but it breaks backwards compatibility. Some existing code depends on pointer values of &T being equal or not equal.

As an addendum, LLVM can automatically perform the “&T-as-T” optimization (without inlining) in some cases where the callee function is in the same compilation unit and known to not care about the pointer value. However, these types of optimizations tend to be fragile, easily disturbed when things get slightly complex.

This seems to be an unpopular opinion, but I feel similarly about how sometimes people seem to toss out `inline` (and even more suspect, `inline(always)` annotations on Rust functions like candy on Halloween and there are almost never any sort of actual measurements of whether it actually helps in the cases it's used. It's not even that I think it really hurts that much in most of the stuff I've worked on (which tends to be more sensitive to concurrency design and network round trips), but I can't help but worry that people using stuff like this when they don't seem to fully grasp the implications is a recipe for trouble.

Yeah inline is an absolute classic for this. The number of uncommented __attribute__((always_inline))s I see in C code drives me crazy. There are absolutely legitimate reasons to use that attribute but there should ALWAYS be a comment about why, so that later readers know in what conditions they can safely remove it.

In the old days I saw C code with massive overuse of the "register" keyword. Meaning this variable should be kept in a register. Someone had ove 12 of these in one function back when that was waaaay more variables then processors had registers.

Good thing modern register allocation by compilers makes this irrelevant.

As I recall, "register" was already a no-op in C compilers by the 90s or so (maybe except for embedded? I just don't know enough about that).

Even if you do benchmark something, maintainability can be more important than a marginal performance improvement.

I've seen this happen a lot with JavaScript, particularly in the last 5-10 years as JS engines have developed increasingly sophisticated approaches to performance. Today's optimization can be tomorrow's de-optimization. Even given an unchanging landscape of compiler/interpreter, tightly-optimized code can become de-optimized when updated and extended, as compared to maintainable code that may not suffer much performance degradation upon extension.

edit: I misread the previous post. Ignore this.

How are you using the word simpler? Because to me that implies a combination of more obvious and number of lines of code. Something that a benchmark shouldn't be involved in.

For example asking someone to delete 10 lines of code and instead use go's ` net.SplitHostPort` would be an example of "simpler".

I've read that good generals worry about tactics and great generals worry about logistics.

Good programmers play code golf, great programmers write readable and maintainable code.

Your example seems reasonable but programmers also like to act like the smartest one in the room. I often come across tricky and borderline obfuscated code because somebody wanted to look clever. This is a logistical nightmare.

Ugh, you are right but then someone comes and uses this to rationalize not including things like map, filter and reduce in a language because they are supposedly too complicated and you can just do it with a for loop

I work in a Rust codebase that uses a lot of functional functions, and I’ll say this: on average the imperative style takes less lines of code and less indentation. I also find it more readable personally, and idiomatic.

Functional iteration is good for the same reason we use for loops over while loops, and while loops over goto: they are more constrained, more clearly communicate intent, and are therefore easier to reason about.

Sure but it’s easy to go overboard with this stuff. Reduce (fold) especially can be pretty hard to read in hairy situations.

My general rule is that if you need fewer lines of code to implement your logic with a simple for loop, you probably should.

Yeah, I agree with that. Especially reduce/fold, which I find is almost always better written as loop. Filter would be a good example of the opposite for me: almost always much clearer written functionally.

I find I frequently use a combination - use map/filter to setup an iterator, and then reduce it in a loop.

Just because we’re on the topic of performance: the rust optimizer can sometimes generate better code if you use map / filter / etc. The slice iterator in any context is a huge win over manual array iteration because it only needs to do bounds checking once.

Javascript (v8, last I checked) is the opposite. Simple for loops almost always outperform anything else.

I've seen cases where an iterator was better, but I've also seen gains from using an imperative loop with manual indexing. Loop conditions and the occasional assertion can be enough to elide bounds checks. (Though sometimes the compiler gets too paranoid about integer overflow.)

Most of the time you should just write whatever's clear/convenient but sometimes it's worth trying both and scrutinizing godbolt.

This is premature optimization imo. Unless you’re using par_iter

Without knowing the domain, you have no way of knowing that.

It’s also much easier to stick to for() loops in javascript as you code than it is to rewrite everything later when tuning for performance. If that’s something you expect to need to do.

I suspect this has more to do with the lack of first-class sequence comprehensions in the syntax. If you had to write imperative style, but all loops were HOFs, it would hardly be ergonomic, either. OTOH a good query language is much more readable.

That's what they're saying: if someone says anything more complicated is faster, they challenge them to benchmark it. Usually, it turns out whoever argues the "is faster" point doesn't bother to benchmark it and the simpler code-wise thing wins out. So yes - the benchmark goes to performance, simplicity is in lines-of-code, cyclomatic, "in the eye of the beholder" or whatever other metric you choose, but usually it's obvious.

> I always ask them to either prove it or write the simple thing

This is cumulative thou, 1 may not make much difference. 100,000 of those will. There is that story of chrome slows down that was does discovered to be in strings each one of which is too little to make a difference

> I always ask them to either prove it or write the simple thing.

Even if they do, they also need to make a case that in this specific case, performance matters enough to pessimize code simplicity and maintainability.

Also, if performance is that critical, it's imperative to benchmark again after each compiler release to guard against codegen regressions. And benchmark after changing this piece of code. Otherwise, we can say that performance doesn't really matter.

I think you're missing the point of the previous comment, they are saying a good proxy for it being worth it to optimize, is if you're willing to benchmark it.

My favorite way of thinking. Should be applied to question the need for the existence of the given feature or function too, and best to delete the whole stuff.

Reply on news.ycombinator.com