I've seen my fair share of bugs from the compiler incorrectly hoisting variables from loops, mis-using registers, etc. I prefer to turn off compiler optimization as one of the first steps in a new project, but that's me.
If execution efficiency is not a concern and you're using C, then you're most likely using the wrong language. Especially for "business apps".
One shouldn't need to throw the baby out with the bathwater (-O0) in order to get some semblance of semantics that won't pull the rug under your feet when you're not looking.
Undefined behavior essentially says that compilers don't have to care about what happens in the cases that would constitute undefined behavior. This doesn't manifest in the compiler as if (undefined_behavior()) { destroy_users_code(); }, contrary to popular opinion. It instead tends to manifest as logic like "along this control-flow path, this condition is true, so we can now thread the jump from block A to block C since you're redundantly checking a known-true condition" and only after unwrapping several layers of computed assumptions do you find the "we assumed overflow cannot occur" at the bottom.
There seems to be some idea that there is some "evil pass" in the compilers which looks for undefined behavior and then maliciously optimizes surrounding code to do something the programmer didn't expect.
Not at all. Even all the examples of UB leading to unexpected optimizations usually involve a chain of events, with very straightforward and necessary optimizations being involved, like value propagation, inlining, dead-code elimination, etc - and these aren't inherently related to "exploiting UB". You could (in some compilers) disable one of the things in the chain and perhaps avoid the problem for that example, but you'd also hurt a lot of other code that relied on the optimization.
That's one of the problems with people asking for specific small snippets of code where the UB-related transformation produced a big gain: you can certainly find these (but they'll often be picked apart) - but the bigger problem is not with the specific examples: it's that whatever optimization optimization you disable to make the small example work as you'd expect might then produce worse code across your application.
So people who want to disable the optimization that does "that" are often incorrectly assuming there is a small simple optimization which leads to "that" in the first place.
Still, I definitely agree that the situation regarding UB is depressing in many respects. Many of the decisions made by the C committee in the past haven't aged well. If you take a look at the low-level optimizations afforded by the largely-deterministic Java specification, they are largely at the same level as C - but Java had the benefit of coming along a couple of decades later where many open questions in C's time, such as integer overflow behavior, integer sizes, shift behavior, pointer models, etc, had largely been resolved. Platforms that don't conform to the JVM's model of an ideal machine will just have to generate slow code in some cases.
Examples:
I don't care at all about optimizations that are a result of treating signed integer overflow as undefined behavior. I'll go for predictable, deterministic behavior every single time.
Same for strict aliasing rules. I find it absolutely insane and mind boggling that -O2 enables strict aliasing amongst who knows what else (50+ other flags). Why can't the impact of these optimization flags be easier to deduce? Why do I feel like I need to be a compiler developer just to get some measure of confidence in what the optimizers are doing? It's insane that there are people who treat this sort of unwarranted, dangerous complexity as a rite of passage and don't push for something that's better. Most importantly, it's terrifying that a significant chunk of C programmers _are not even aware of these issues_.
C will stick around for decades and will continue to be picked up by newcomers. It doesn't have to stay as dangerous and uncompromising as it is now.
EDIT: There is some progress with things like ubsan and asan but alas they're fairly limited platform-wise. What's worrying is that there hasn't been a clear shift in the mentality of those who control the language as the OP indicates in his report.
Exactly, even if most of us don't touch it directly, we need to use systems that make an heavy use of C (including Objective-C and C++), so it would already be an improvement to our computing stack if C could be improved as such.
And how can I, as a compiler writer, know what you, the compiler user, consider acceptable without reading your mind?
It would be nice if we had some sort of contract. I, the compiler writer, will say that, so long as you write code that conforms to this contract, I will faithfully execute that code. And you, the programmer, will say that you will only write code that conforms to this contract, and that I don't need to worry about you writing code that fails to conform to this contract. Well, somebody already wrote this contract: it's the C specification.
What you're really saying is that you don't like the contract you have, which is a totally valid position to take (I myself would love to see strict aliasing go die in a fire). But almost no one who objects to the C contract is willing to actually negotiate a new contract. Instead, it's almost invariably a complaint that amounts to "you fucking compiler writers are ruining my code with your stupid optimizations." Well, no, your code was already broken per the contract; we were just too dumb to realize it earlier. And for the big undefined behaviors where well-behaved semantics are feasible (such as strict aliasing or signed integer overflow), we provide options to let you say "I don't agree to the C contract, I want to agree to a not-quite-C contract instead."
Many people right now don't realize that undefined behavior is merely a contract that the programmers not to create these statements, true. But that's why we, as compiler writers, try to educate people about why undefined behavior exists, what it means for programmers, and providing tools to make it much easier for programmers to find where it exists (hence things like asan and ubsan).
The more layers of abstraction, the worse the penalty for not optimizing, so C++ is usually more heavily affected than C, for example: many of the so called "zero-cost" abstractions in C++ rely on a good optimizer.
Two things.
Thing one: Execution efficient on super scalar processors is very unlike what you assume it to be. For instance you assume alignment makes you program fast, when in fact it makes it slower due to cache pressure.
Thing two: On lower end processors what's important is not speed but code size.
Thing three: (off by one error natch) The amount of critical C code is tiny tiny tiny compared to the amount of C code where execution speed just does not matter because the code is almost never executed. The bleeding hot sections are often rewritten in assembly (encryption cough decryption)
It's not. What do you think I "assume execution efficiency to be"?
> On lower end processors what's important is not speed but code size.
That's why I said "most likely", not "always". Besides, those tiny microprocessors typically don't run the "business apps" which bokglobule was talking about.
> The amount of critical C code is tiny tiny tiny compared to the amount of C code where execution speed just does not matter because the code is almost never executed.
Not true for most software. Sure, most applications spend most of their execution time in a small part of the code, but it's typically not so small that you could easily rewrite it in assembly.
> The bleeding hot sections are often rewritten in assembly (encryption cough decryption)
Cryptography kernels are written in assembly to ensure that they have a constant execution time to prevent timing attacks. Not so much for performance.
You assume that modern high performance processors execute instructions. They do not, they analyze, optimize and emit their an internal instruction stream and then execute that. That extra step is what makes fiddly UB type 'optimizations' worthless. And since you don't know what processor is going to execute the code and how it's optimized, the speed gains are basically 'noise'
> Besides, those tiny microprocessors typically don't run the "business apps" which bokglobule was talking about.
"Business apps" is usually network and io bound more than anything.
> Not true for most software.
Dan Bernstein says you're wrong.
> Cryptography kernels are written in assembly to ensure that they have a constant execution time to prevent timing attacks. Not so much for performance.
Dan Bernstein says you're wrong here too. Performance is everything with encryption. Consider video streaming over and encrypted connection. Yeah that.
Yeah, stop making stuff up about me.
> they analyze, optimize and emit their an internal instruction stream and then execute that.
I know how an out-of-order, superscalar processor works. I also know that they can't do magic, because their optimizations are severely limited by time and scope constraints (although they do have access to some information that isn't available at compile time).
> That extra step is what makes fiddly UB type 'optimizations' worthless.
I'm not arguing for optimizations based on UB. This subthread is about compiler optimizations in general, which bokglobule claimed to be nearly always unnecessary.
> Dan Bernstein says you're wrong.
Do you have a study you can cite? Do you think just dropping a name will convince anyone?
> Performance is everything with encryption.
I never claimed it wasn't. I claimed that performance isn't the reason why cryptography kernels are rewritten in assembly, and that's because C + optimizing compiler is already fast enough and the small performance gain alone doesn't justify the switch to assembly.
i might be with you on the former, but that first pass really cuts down on executable (icache) size, nets a huge performance win and actually makes the generated code alot easier to read.
some of those analyses can get pretty involved. but i dont think we should be discouraging people from investigating them - or from providing nice safe defaults and flags to turn them on.
one other speed-independent factor here is that like you I usually start without any optimizations. and usually when I turn it on I find a few bugs in my own code right away. thats pretty helpful.