Reducing the Performance Impact of Debug Code in C Libraries

Replying in reverse order to your questions:

With the debug_on checked moved outside the loops there will be no performance difference. The point of the loop is to emulate paths that are called many times and can't be optimised by the compiler, for example, having the debug check inside a different function that does something else.

I'm not sure if it's the check itself or bad branch prediction that caused it, it doesn't matter for my case, as it's still a cost. It also happens when I use GNU extensions (__builtin_expect) to mark the test as unlikely to be true.

Was it the boolean check that degraded performance or was branch misprediction inside a nested loop the root cause?

The solution you came to seems quite elegant but I'd be curious to see metrics with the debug_on check moved outside both loops.