Hacker News Reader

Comment by brancz | Hacker News Reader

I happen to work on profiling tools where we rely on compilers to output metadata about the machine executable code correctly. It turns out this is surprisingly hard and goes wrong quite often, and sometimes it's even impossible to maintain correctness across multiple rounds of compiler optimizations. For us the way compilers solve this is basically the worst possible scenario, when the compiler can't correctly track it across optimizations, it just starts emitting less or sometimes even no information. These bugs are just awful to understand.

At the end of the day compilers are also just software and it's just a matter of how many features you end up being exposed to.

View on HN

The sort of thing profilers and debuggers want doesn't really exist at high optimisation levels. In my experience that kind of thing isn't exactly a bug, it's just that the information you want can't even exist in theory.

Kind of the same thing as reporting memory usage for programs. You want to be able to say "A uses 3MB, B uses 2MB", but then there's shared memory, file caches, fragmentation, etc. So much complication that "how much memory is this process using?" isn't even a valid question anymore.

Debug info under high optimization is very tricky. A simple example:

for (i = 0; i < 4; i++) ...

If the compiler unrolls the loop four times the eliminates the induction variable altogether, then combines the four iteration loop into a single vector instruction, what debug info should it emit for the value of i in the loop body?

You can argue that it should emit what i would have had at a given point, but now you have four different values for i all at that one vector instruction. Reasoning about that is quite difficult.

True, but I would be happy if at least the debugger could point me at the value which is clearly being iterated over in a register, or tell me the value of a standard ABI function's arguments (and no, it hasn't been overwritten by something else).

> the information you want can't even exist in theory.

There are things that do exist, though, right? For example, the compiler knows which values are live at every point of the code, and in which registers and stack slots they are located. In most cases it even knows the SSA that produced them, and—again, in most cases—that SSA is expressible in the source language in terms of user-visible values. (How to display that ergonomically in the presence of inlining is a problem, though.)

None of this will fit in DWARF, but honestly DWARF is enough of a slog to work with that I’m not really sorry about that. (And PDB is not a contender until actual first-party documentation and library code exists.) The design work is thorny, don’t get me wrong, but I don’t think this is impossible.

> For example, the compiler knows which values are live at every point of the code,

No because "point of the code" is not well defined after optimisation. The compiler can reorder statements, split them, merge them, inline them, outline them, eliminate them entirely etc.

The compiler needs to know that information in order to do said optimisations, so it does have the information somewhere, it just fails to express it in the debug information.

I meant each point of the machine code, of course.

You're 100% correct, but for our users it sure seems like a bug.

Reply on news.ycombinator.com