Comment by obl | Hacker News Reader

Comment by obl | Hacker News Reader

Parent Full thread

You should read the post you are replying to again, as it is well written and precise.

Basically, the intent of the proposal is understandable but it does not make sense as written.

The spec does not explain how to compile C code, it gives you a set of rules so that you could in theory take some piece of C source and a full execution trace and verify whether that trace was a conforming execution for that C code.

You take the example of the negative shift again, and it is addressed in the post you respond to : it's a perfectly fine proposal to demote negative shifts from UB to something less permissive to the compiler.

However, if you do so, of course, you have to frame what behaviors are accepted for a program doing such a negative shifts, just as you did in your post !

So let's say : arbitrary (but consistent) values are fine, termination is fine. That's great because it means we can still compile a shift to a CPU shift and there's no UB anymore. Ok.

Same goes for signed overflow.

However this is a case by case process, and you will absolutely not be able to ascribe meaning this way to many forms of UB. For example, as the GP said, storing through an invalid pointer.

If you want to compile a pointer store to a store instruction without runtime checks, then possible observable behavior include : other variable changing values, function call not returning to their calling context, and even of course running a full ROP chain exploit !

It's no surprise that in that case the specs says that the set of allowed observable behaviors is anything.

Argument like yours seem to rely on an incorrect understanding of the design of C. Languages like Java encapsulate program execution in a runtime that, in theory, can catch and manage all sorts of programmer errors that C does not catch. That's ok. If the programmer references past the end of an array, the compiler may catch and object, it may simply compile the instructions and let the program step on its own data structures or encounter a memory fault or even a protection violation if the architecture provides some type protection. And it's bizarre to see this warning about ROP exploit, when current "optimization" based on UB is a source of ROP exploit vulnerability. One cannot reliably put protections in code against bad pointers if upstream errors are grounds for deleting the guards.

> It's no surprise that in that case the specs says that the set of allowed observable behaviors is anything.

And yet UB is frequently described as this scary monster that will eat your hard drive.

Sure there are more UBs in C than there would need to be if the standard restricted more on contemporary hardware. I think it still allows ones' complement representation for example? But I guess it shouldn't be so bad as long as compilers do not exploit the freedom to do anything that comes from UB, more than they need to on a particular architecture?

Compilers do not "exploit the freedom to do anything that comes from UB", they just assume that the program does not have UB. This might or might not lead the compiler to behave as if it was doing anything it wants, but it's not screwing the programmer on purpose.

As for "UB is frequently described as this scary monster that will eat your hard drive", setting aside that UB could be exploited by an attacker, it's not hard to imagine, for instance, a recursive file deletion routine that is affected by UB misbehaving and erasing more than it should.

Let me take a shift operation as an example. On architectures where it's no problem to "overshift" (e.g. the operation always results in 0), the compiler could be just fine and leave shifts alone. Maybe a 0 is exactly what the user wanted!

Or, the compiler could assume that the programmer made sure that overshifting never happens, since it's UB. Based on that assumption, it could make more aggressive (and possibly totally insignificant) optimizations by inferring other assumptions that don't align with the programmer's idea anymore.

I never encountered such a sitation myself, but there are stories like that on the internet, where for example the Linux kernel was broken.

Another scenario, the user might always do an operation that sometimes results in a random value (which is UB), but in the end only use the resulting value if it was not UB. The compiler instead could assume the operation can never happen in the first place. Again, there is a misalignment here in how the compiler reasoned and how the programmer reasoned (closer to the actual hardware).

So, the difficult problem is probably defining the UB configurations as narrowly as possible. It might be a good idea to specify UB per architecture in some cases, but then it would lead to a lot of complexity. In any case a guideline for compilers should be to minimize surprises, which means not applying many optimizations around UB. That's easier said than done I guess.

> As for "UB is frequently described as this scary monster that will eat your hard drive", setting aside that UB could be exploited by an attacker, it's not hard to imagine, for instance, a recursive file deletion routine that is affected by UB misbehaving and erasing more than it should.

I'd figure it's much more likely to happen as a result of a logic error introduced by the programmer, without any UB involved.

There's also this infamous example of UB: https://kristerw.blogspot.com/2017/09/why-undefined-behavior... (explanation: https://blog.tchatzigiannakis.com/undefined-behavior-can-lit...)

Thanks, that's interesting. And still, it's artificially constructed, so concerning the point I made (actual logic bugs are more likely) I think it stands.

IMHO this is clearly a compiler bug. Yes, it's technically allowed to do this because calling a NULL function pointer is UB, but nobody wants such an optimization (which is my first point from earlier). If the compiler makes such a decision based on UB, shouldn't it be able to at least issue a warning?

Furthermore, why should this be optimized in the first place? If the user decides to make a function pointer instead of a hard-wired inlineable call, he is aware of the performance implications (which are negligible in 99.9% of the cases). Where function call efficiency matters, nobody ever used a function pointer.

Shouldn't the compiler architecture be able to avoid making this optimization, simply by avoiding propagating the UB? It should simply not say "Calling NULL can never happen". This way it would never make optimizations that make the situation even worse. I think this would even be less optimization code!

Instead, when the UB happens, the programmer should get what he deserves (which is, well, calling NULL, and most likely resulting in a segfault, which is the right thing).

On the other hand, if the compiler can infer the value of a function pointer based on propagation of values alone, I'm fine with hard-coding the address, even though it's probably not worth the effort anyway.

That's a "devirtualization" optimization. Suppose you have a C++ class with a virtual method (or something similar coded in C), which only has a single concrete implementation. Clearly, every place which calls that virtual method through any pointer to that class can only be calling that single concrete implementation, so the compiler can call the function directly instead of through the function pointer, perhaps even inlining the called function.

Yes, that EraseAll example is silly, but how can the optimizer distinguish it from non-silly examples? Perhaps there are other alternative functions which also set that same function pointer, but they were all hidden by some #ifdef; in that case, devirtualizing when there's only one option left will be a performance win, and can expose other optimization opportunities.

> devirtualization

Hmm, shouldn't it be possible to do that in some other way? After all, a virtual function is something different than a function pointer (the vtable holding a function pointer is only an implementation detail, isn't it?).

Anyway, I'd rather not have that optimization. When we don't want to pay a small performance hit, we shouldn't make a virtual function in the first place. It's good to see how advanced optimization tricks can combine in dangerous ways with complicated language features!

> distinguish it from non-silly examples?

Yes, probably not possible. There is a real risk, but non-silly examples are less likely than constructed ones.

> all hidden by some #ifdef

If you do it with the preprocessor, there is no need for function pointers in the first place. Just make a #define instead of a variable.

Thanks again for the example, very interesting!

Reply on news.ycombinator.com