Hacker News Reader

Comment by ryandrake | Hacker News Reader

One of the things I'll probably never understand is people's discomfort with and/or refusal to familiarize themselves with the use of a debugger and stepping through code. People will spend hours saturating their code with logging, rebuilding and running the code, and then spend more hours pouring through the logs as if walking around with a divining rod trying to guess what's actually happening, when they could simply set a few breakpoints, examine the variables and control flow as they're happening. The biggest "superpower" to my programming skills as a junior engineer was learning how to use the debugger well.

Yes, there are bugs that don't reproduce in your own environment, and yes, logs are a good starting point for those bugs. But, as a general tool to reach for? Not a big supporter of print() debugging.

View on HN

One problem I have is that I primarily work on embedded distributed systems (multiple software components talking over a high speed messaging layer, using both sockets and shared memory).

As soon as you pause the code in a debugger you "disturb the equilibrium" in such a way that debugging the original problem becomes impossible because remote components start timing out or closing sockets, etc.

I still end up debugging via print statement a lot so I can run the code at full speed. Does anyone have a good tooling suggestion for a situation like this?

My dream is a tool that would allow me to set a breakpoint which would start capturing program state at each line while still running at full speed, but then allow me to step through the recording later. Does something like this exist?

Yes, it's called rr. https://rr-project.org/

This is very cool. Thanks for sharing it.

GDB can kind of do this, you can have commands fire when a breakpoint is hit and continue automatically, and if you want to take it further you can use Python and execute Python code to inspect the state of the CPU and memory in some programmatic way. Not sure if it applies to your target platform, but if it’s ARM it should work.

For me, the most annoying part about a debugger is that it always shows a snapshot at a single point in time, and the snapshot can only march forward. A good fraction of bugs in my code are ones that surface gradually or occasionally over a long sequence of operations. Stopping at the ultimate failure point is far too late, and breaking at every single iteration is far too tedious. But with a log of all the iterations, I can trace the error back to the initial point where the program no longer acts as expected.

That is, I very often find a narrow view of the program state over the whole computation to be more informative than a wide view of the program state at any particular moment in time.

In some cases, GDB allows reverse execution of code:

https://sourceware.org/gdb/current/onlinedocs/gdb.html/Rever...

https://rr-project.org

This is indeed a problem people have with debuggers, so some very smart people found a way to fix it.

rr is so good. "Where was this variable written? Where did this value come from?" Set watchpoint, reverse-continue, and bam you arrive at where the write happened.

Indeed, that fixes the part about it only marching forward. But still, it wouldn't be sufficient in cases where the ultimate cause is near neither the start nor the obvious error, but (e.g.) in the 50th of 100 entries. At least, not unless the initial error is simple enough to be captured by a conditional breakpoint. But even then, I'd probably be reaching for my prints just to determine what a good condition might be.

We'd be hosting the RR inventor to a tech talk soon! I'll post a link here when we have it scheduled

> Indeed, that fixes the part about it only marching forward. But still, it wouldn't be sufficient in cases where the ultimate cause is near neither the start nor the obvious error, but (e.g.) in the 50th of 100 entries.

Not true. rr (and time travel debugging in general) really excels at exactly that problem. You just put a watchpoint on the offending data at the 'obvious error', and reverse-continue. This usually takes you straight to the root-cause. It's incredibly powerful.

(Disclaimer, I'm not exactly unbiased as I work on another time-travel debugger, https://undo.io)

Because logging is persistent. Once I have good logging implemented for a section of code I can always enable it and see what's happening every time the application runs.

Debuggers are one-offs. Running a debugger (and rerunning the application) every time I'm suspicious about something, or need confirmation, is more effort.

Good logging is having an always-on debugger that's enabled for everybody at once. It's an investment that pays off.

Yes, I know how to use a debugger, but rarely has it been more valuable than good logging.

I do both.

Almost all the inflection points you would want to print to debug, it should really be a debug log level statement that stays in the code. If you're writing a lot of these you might want to simplify the code.

Otherwise yes use a debugger, but question why you're having to debug in the first place. Improve the code quality while you're at it instead of just fixing the bugs.

This is probably going to sound weird, but debugging feels a lot like washing dishes to me. Do more with less and avoid toil.

Logging is just easier. The lazy route is the most common

Reply on news.ycombinator.com