Hacker News Reader

Comment by sanxiyn | Hacker News Reader

> I've seen arguments that some people desperately need every last bit of performance, and even a few cycles inside a loop is the difference between a viable product and failure.

This is a strawman. The usual argument is that 1% better performance is, well, 1% better. That's a whole server if you have 100 servers. People are not desperate, but do want an additional server for free.

> Assuming that's true, then they should be crafting assembly code by hand.

This, and "a few cycles inside a loop", is another strawman. What usually happens is a good compiler saves a bit everywhere, which helps when the program is optimized to flat profile without hotspot. Since saving is everywhere, "crafting assembly" amounts to lots and lots of assembly, not just a few in strategic places.

View on HN

To add to this, performance is very important in certain domains.

Take game development for instance, if your code runs a bit faster that's more frames you're rendering per second. This can mean the difference between a laggy, unplayable mess and a good gaming experience on an average machine (a laptop, or a non-gaming rig).

Most software is not games or scientific computing. Most software is not speed critical. This is why most new software is not written in C.

Most new software does run on devices with batteries though. Having the CPU sleep more is a good thing to have.

This is an interesting thought.

Do we know that most new software runs in batteries? Do you have a reference for that? It would be useful.

Other question is - does a sleeping CPU always win, or are other optimisations such as memory usage, screen usage, and communications, more important? I know for sure that the CPU is not the main consumer of power on a bunch of Android phones that were used in an academic study.

I don't have hard numbers. Most computation is probably done on servers. But power matters there as well (a lot). Most devices sold today are laptops and smart phones, I think. Some people claim that the IoT will be a thing real soon now, with embedded sensors everywhere.

Probably the CPU is not the main power sink on mobile devices (screen and radio are I think). But cranking up the optimization level is free for the developer.

So, this was my point - is it really free for the developer? For example, GPGPU compilers full of subtle bugs cost significant development time.

GPGPU is still relatively new and the tooling is still under work. But compiling with g++ -O2 instead of -O0 is pretty safe

Not all software need to be speed critical but all softwares benefit from being faster. Part of the reason why we can afford to be less optimal when writing software is because many libraries have already been optimised for us (and it's not uncommon for these libraries to map to C or assembly code to utilise the hardware more efficiently), ranging from serialisation libraries to crytography libraries. This is not to mention the OS that we run the software on and other tools that the software depend on (such as databases). A small compiler optimisation improvement across the entire ecosystem can have a dramatic effect.

> but all softwares benefit from being faster.

It's not so simple. If you're trading off simplicity in the compiler for that speed,for example, it may be a net loss.

If speed was the only concern, we'd all go out and buy super computers.

Both chrome and firefox lag all over the place on a modern quad core box with 16GB of ram and hardware accelerated rendering on a 8GB r290. Most software might not be speed critical, but those few milliseconds not saved add up quickly, and performance should definitely be a goal of most (end user) software. I mean if you're only running the software on your own servers, then by all means ignore performance as much as you please if it's a worthwhile tradeoff for you. But please don't ignore speed when it comes to software other people have to run.

Chrome and Firefox suffer from having to run other people's (likely unoptimised) software in their sandbox.

You can optimise as much as you like but then John Q Madbugger comes along with their insane Javascript (not looking at Flickr here, no sir) and suddenly your performance is in the toilet.

Yeah, the bloatloads of javascript are probably the worst offenders of ignoring performance of software other people have to run, sometimes even reimplementing features the browser already has, just slower and worse (Listen Zendesk, my browser has been doing tabs just fine for a decade, implementing them in javascript and entirely hijacking normal browser navigation behavior doesn't make anything better for anyone).

Thinking over it some more, browsers themselves were probably not a good piece of software to call out on performance considering how much optimization is actually done to increase their performance. The horrid things people do on top of them deserve all the blame they get though, and then some.

The QNAP web interface implements a whole internal windowing system in Javascript. It is the stupidest dumbest nonsense I've seen for years and is blitheringly painful to use.

And a good compiler doesn't only optimize for speed. It also optimizes for code size, memory usage.

I do however agree with underlying premise of this post: Code that works 100% of the time is better than higher performant code that explodes in your face every once in a while.

> Code that works 100% of the time is better than higher performant code that explodes in your face every once in a while.

That depends. In game programming, if I had a choice between a render that will crash once every 10 hours and produce 60 fps (in some model envrionment) vs a render that never crashes but produces 45 fps, I would choose the first one.

Ditto for math libraries, machine learning libraries + tools, statistics calculations and matrix multiplier kernels.

Hell, I'd take a 1% chance of blowup for a 10% speedup any day.

> I'd take a 1% chance of blowup for a 10% speedup any day.

Are you serious? Would you base your research career on software that you don't trust? I wouldn't.

Please, notice that 1% chance of full blowup means a not-null probability of nice-looking-but-actually-false results from time to time, and that's a risk no researcher is allowed to accept.

> Are you serious? Would you base your research career on software that you don't trust? I wouldn't.

I guess you haven't seen the quality of academic code recently. The choices are basing your credibility on slow badly written code or fast badly written code.

That is what we have fast math for :)

>And a good compiler doesn't only optimize for speed. It also optimizes for code size, memory usage.

No they don't. Code size is rarely a problem (embedded probably cares..). Optimizations will in most cases lead to larger code, loop unrolling, inline expansion etc. In c the compiler cant't, in most cases do anything about memory usage. (Probably could pack structs.. , but that is not something the compiler would do I think, you would break ABI with non optimized code). In managed languages, memory usages is probably more depended on the runtime system GC, then compilers.

GCC has a flag for code size optimization -Os https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html

From that exact link: "Turning on optimization flags makes the compiler attempt to improve the performance and/or code size at the expense of compilation time and possibly the ability to debug the program."

"With -O, the compiler tries to reduce code size and execution time"

So actually yes, The compiler optimizes for both size AND speed. Sometimes you can optimize for maximum speed (-O3), and sometimes optimize for minimum code size (-Os). You can even optimize for best debugging or best compilation times, and forget about size/speed.

As for memory usage -

1. First of all - code uses up memory. smaller programs = less memory used for holding the exectuable code.

2. Consider the various ways you could compile a switch(-case) statement. You could build it as a set of conditional branches one after the other, you could compile it into a binary tree search (or a hash dictionary lookup), and you could jump to an address found in a lookup array table.

This former method is also called a "branch table". https://en.wikipedia.org/wiki/Branch_table

A branch table would be the fastest method for implementing a switch with a few sequential values (0,1,2,3,4,5) But would be a really bad idea for sparse values (0,200,5000,10000,200000).

Choosing not to use a branch table for sparse values is an optimization for Memory usage. And compilers make this decision all the time.

> That's a whole server if you have 100 servers. People are not desperate, but do want an additional server for free.

Renting an extra server costs practically nothing comparing to wage costs for a tech company.

So I'd say they'd care more about the compiler being reliable and predictable.

Maybe when it's 1 server.

I have a team that regularly generates savings of XX% of the entire datacenter CPU cycles of a very large fleet through better compiler optimization (that pretty much never breaks any of our programs. In fact, we break them more through almost anything other than better optimization. Shocking, i know, but you can test this stuff really well it turns out, and write sanitizers to find behavior you wouldn't discover at compile time and ...)

I know at least another 100 large companies where the cost savings of doing this kind of thing also totals in the tens to hundreds of millions, per year.

and that's just the normal X% a year, not the "we put a team of experts targeting your use case and they got you 100x" type of savings.

So yeah, there are people for whom this matters ... a lot, and dismissing them casually is not a great idea :)

Truthfully, IME, compiler-optimization induced bugs are so infrequent compared to normal you-wrote-really-wrong code bugs that IMHO it's missing the forest for the trees. (now, maybe people suck more at tracking them down, but ...)

Heck, I had once screwed up some code in GCC that caused it to essentially remove random stores and loads from the program being compiled. It only caused one visible failure across a bunch of real world apps that had serious validation suites.

From reading hacker news, you'd get the idea compiler optimizations follows your code down alleyways to mug it. In truth, people are so busy overflowing buffers and whatnot that we are lucky if we can break their code due to optimizing.

How many software engineers are working on data centre scale systems?

Performance gains of 1% just don't matter to most programmers.

I don't disagree with the above, but I do see some circumstances where dropping down a carefully hand-tuned page-long assembly loop is warranted. It's not something I'd recommend in the general case, and only if the problem is likely to remain completely well specified for a long enough time to pay off. That is not generally the case when the assembly blob starts getting appreciably larger than a single loop, e.g. crosses function boundaries, needs to coordinate complex data structure layouts, etc.

> Assuming that's true, then they should be crafting assembly code by hand.

This guy, what a fucking joke. Yeah to avoid bugs instead of using these fancy compilers (which rarely if ever are the cause of bugs) we should be assembling our programs by hand.

I've seen people go to extreme lengths to teach compilers particular optimizations that only apply in extremely narrow circumstances. Worse, they usually end up additionally have to introduce a way for a compiler to assume invariants that would break normal programs (e.g. overcome weaknesses in alias analysis). I've seen people go to extreme lengths and rely on C++ undefined behavior to get some weird quirk out of some specific machine instruction. All of that is extremely brittle; move to a new compiler version, new compiler, or even slightly alter the code, and suddenly the optimization either stops working or the compiler generates wrong code.

At some point, for extremely hot loops, there may be no choice but to write a few pages of assembly by hand. If this saves tens of thousands of lines of compiler code that doesn't generalize, and is used on thousands or millions of machines, it's worth it, IMO.

Reply on news.ycombinator.com