On Using Debuggers

> "It saves me over and over again when dealing with code I don't understand."

This is the key statement of the original post.

If you don't understand the code, then you need to sit down and read it until you do.

Using a debugger can lure you into quick, localized fixes to make small problems disappear. If you just need to make a minor fix to an external library that you'll never look at again, this is probably OK. But if you own the code, you're in a rut.

Constant use of a debugger is a crutch that prevents you from deeply understanding a problem or discovering core issues.

In the long run, a debugger slows you down. Don't step through code to verify fixes; write automated tests. Don't set breakpoints to verify that a value is what you expect; write assertions to confirm that they will always be so. Don't step through new code to verify it is correct; write a unit test first or compose your new code from smaller, proven pieces.

When confronted with a bug, sometimes the best thing to do is just stare at the ceiling and think about what might be causing it -- quite often you'll find the right answer.

Some bugs -- the really hard, nasty ones -- may never be revealed by stepping with a debugger. Performance problems aren't shown by a debugger. Only understanding your system at a high level can inspire you to make fundamental changes in your code. A debugger blinds you to these.

[Full disclosure: I make heavy use of both debuggers, and printf debugging, in my work.]

A debugger gives you a detailed picture of the state of your program at one moment in time, or (when stepping) over a short period of time. Other tools -- logging, monitoring, etc. -- provide a different slice through your program state, making it much easier to observe the flow over time, but relatively little detail about any given instant. Each approach has its uses.

Logging has a number of properties that make it relatively more useful for experienced programmers, mature codebases, and/or "infrastructure" systems (compilers, operating systems, libraries) that are used far from the developer's workstation. That might explain its popularity among the group interviewed in this book. For instance:

1. Programmability. The nature of logging -- both generation and analysis -- lends itself to building up a library of utilities. An experienced programmer develops a toolkit -- part code, part knowledge -- that makes logging more and more useful to him/her over time. There's also a learning curve for debugging techniques, of course, but it flattens out sooner. I've been programming for 35 years, and my logging techniques are better than they were even 5 years ago, but my productivity in the debugger plateaued decades ago.

2. Accumulation. If you work on a single body of code for a long time, you can continually improve the usefulness of its logging. Eventually you reach the point where many bugs are quickly obvious from the log, because you'd previously invested a lot of effort in logging exactly the data you typically need, in the most convenient format. Printf debugging can take a long time if you're starting from scratch (the program contained no relevant logging to start with), but in a mature codebase where attention has been paid to logging, the story can be very different.

3. Universality. Logging tends to work in a wide variety of environments, from manual testing on the developer's workstation, to production servers, to installations on a customer's computer. If you spend your life tracking down hard-to-reproduce bugs reported by other people, logging is essential.

4. Repetitiveness. Tracking down problems in a debugger tends to involve a lot of boring, repetitive actions -- "aha, this is null because that other flag had been set, let me restart and walk through the whole thing again to see when that happens". It's still sometimes the best tool for a job, but it has a time-sink flavor that probably pushes some people away.

A lot depends on how good your debugger is. If you're coding C# in Visual Studio or Java in Eclipse, the debugger is dead easy to use and works well for solving problems.

On the other hand, the only thing I use gdb for is cheating at nethack (+99 blessed Unicorn horn anyone?) and looking for buffer overflows. There are a few PHP debuggers out there but I've yet to get one to really work right.

My main trouble with printf-debugging in all it's forms is that you end up making changes to code that don't always go away. Even if you do it right 99% of the time, you'll insert 1000's of printf's over the course of a month and sooner or later one of those will end up visible to end users, will fill up your logs with junk messages, or otherwise gunk up the works.

Of course, there's something to say with fixing bugs by looking at the code ~thinking~. Yesterday I was tracking down a bug involving an area of code that I didn't trust, and ended up refactoring it, converting some properties in an interface into readonly methods of an abstract base class so I knew for a fact that certain variables wouldn't change after initialization.

This work convinced me that the code was wack, but when it was done, I knew the problem involved initialization of the objects. For a second I thought about using the debugger, but finding the specific case that was causing trouble would have been tough.

At that point I decided to trace the codepath involved and immediately saw the cause of the problem and understood the root cause (certain initialization code was duplicated in two places, which is just asking for things to break.)

I could have fixed the bug quicker by just following the codepath, but spending some time cleaning up code I was scared of will pay dividends in the future... And it's a good example of the tradeoffs you face when you fix bugs.

That said, I particularly find printf-debugging useful when something strange and difficult-to-reproduce is going on, particularly in a production system.

You should also be careful about taking your opinions on debugger usage from compiler, virtual machine runtime, and operating system implementers. Debuggers, particularly gdb, are seldom useful for more than crash dump analysis when you're looking at situations such as: - Bad optimized code being generated - Crashed trying to do something like execute an integer (happens with GC race conditions) - Deadlock that only happens on maximum number of available cores when no debugger is attached

I've had to deal with all of those specific issues in the last couple of months on our compiler+runtime, and there's sadly not a lot of tools in the free space to help with situations like that.

Queue 1000 comments from people gloating about how they too are "too good for debuggers". I work with a number of people who don't use a debugger, primarily because they never took the time learn how to use one. They often spend hours and hours inserting print statements, recompiling, redeploying, etc. for problems that can be solved in 5 minutes by using a debugger. Print statements are fine if you're debugging your "vending machine" project for CS101, but they don't work well for large, complex projects where you only work on a subset of the codebase. Don't let me dissuade you, though -- you guys keep using your print statements, and I'll be the superstar getting the same work done in a fraction of the time.

Here's a wild assertion about why some programmers - some good programmers - prefer printfs: good programmers prefer to solve problems only once, rather than repeatedly.

When you have a question you want to answer about the state of a program, a debugger gives you powerful tools to answer your questions right now, but doesn't leave any permanent record of your investigation. Each time you subsequently want the answer to that same question you have to run the program, hit that breakpoint again, and cast whatever incantations are necessary to reveal the relevant program state.

If you used printfs instead, you would probably have had to iterate a few times to get the answer the first time; first you print one thing, then another, moving your prints around until they tell you what you want to know. When you're done, though, you've probably left some log output hanging around. So after you're done investigating, say, how the data access layer works, you have something to cross reference while you're poking around the user interface code.

I think this is a more useful technique for unfamiliar source. Comprehending a large hunk of source typically requires integrating your understanding of several sections of code ranging widely across the codebase.

Anyway - a good debugger is a wonderful thing. I can't stand up straight sometimes for leaning on them, though.

When I think debugger, I think gdb, which is fine to a point. But after a while, its interactivity becomes a liability. I don't want to have to set the same breakpoints and watches after a recompile, every time.

The integrated debuggers in IDE's are much better, particularly the Java ones. The C/C++ ones have the problem that they will work fine on code compiled with debug symbols, but on release code they're awful.

And when it comes down to it, the really hard to fix problems are the ones that only show up in the optimized release code.

I find knowing what the stack is at the point where things go wrong incredibly useful. Having an IDE that allows me to work my way back up the stack and see what direction my code was called from that meant that I ended up with an invalid state is terribly handy.

I also sometimes find it useful when calling code I don't have decent documentation for. I can examine the various properties of the object I'm trying to work with, and see which ones contain useful information.

What I don't read much about, but depend on, is using a debugger on new code when there aren't obvious bugs in it. Just walking through it, while viewing variables, gives a wonderful sanity check to verify that code is doing what I think it is doing, and this catches a surprising amount of non-obvious bugs, and lines of code become easier to reason about precisely sometimes when you have visibility of the specific state of the program at that time. This is not a substitute for unit testing obviously, but it can catch problems that wouldn't be obvious or easy to unit test for.

For me, this is also a wonderful way to get accustomed to someone else's code, seeing exactly what it does first before spending the brain cycles upfront to reason about exactly what is going on.

The print statements to debug was a constant in "Coders at Work" too. Also in that book I found interesting that everyone interviewed was asked what the most difficult thing they had to debug was, and almost all of them said something to do with concurrency.

I almost never use a debugger as well (used to rely on it when I was a beginner though). Even when I'm looking at someone else's code, my time is generally spent more productively analyzing it with pencil and paper than running a debugger.

As some one else mentioned, I typically only use gdb for crash dump analysis. The one tool I do use a lot though is Valgrind (to check for memory leaks).

For C/C++ the debugger is an invaluable tool.

For large projects (especially large projects that don't have the files structured well) you can end up with very high compile and link times (> 2 minutes). Using a debugger in this case is much faster than incrementally adding print statements to the code.

Being able to see a callstack is also incredibly valuable on complex projects. The value of printing things out drops dramatically once you have multiple entry points into the same codepath.

Also, if you always run your program through the debugger, when you crash you can immediately diagnose and fix the problem without having to reproduce the crash. This is useful for hard to reproduce bugs.

Besides, it's not like this is an 'either or' type thing. You can use a combination of reading the code, debugging, and printing to track down problems.

I had this conversation about a decade ago at http://www.perlmonks.org/?node_id=48495. Unfortunately most of the links in that discussion have gone dead. Here are fixes to the most important ones.

The Kernel Traffic summary: http://kt.earth.li/kernel-traffic/kt20001002_87.html (sections 1 and 4 are relevant).

Linus' "I'm a bastard": http://lkml.indiana.edu/hypermail/linux/kernel/0009.0/1148.h...

Richard J Moore discussing IBM's experience: http://lkml.indiana.edu/hypermail/linux/kernel/0009.1/1307.h...

I understand this - debuggers are useful (especially when going through code you haven't written), but if you're writing an entire program yourself, you understand how it works completely, so a debugger is much less necessary (not unnecessary).

I haven't fired up a debugger for a long time, partly because I'm in to defensive programming big time. If you have something as simple as assert(ptr != NULL) in a function, if it fails, you get hit with an assert failed message, the file, line, function and condition. Most of the time this results in me thinking, "oh, duh" and immediately correcting the problem, sans debugger.

Using a debugger on pre-production code is a waste of an opportunity to create good tests.

A debugger can be useful when debugging production code, especially when the heat is on. This is no time to stand on ceremony and idealism.

For learning new code, particularly frameworks ime, a step debugger is incredibly valuable. But you are not really debugging in that case.

A debugger is a non essential tool these days. But if the kernel boys said they found it useful, then I'd not argue.

What I tell others: I'm so good that I don't need a debugger.

The truth: Print statements work so well that they dramatically decrease the need for a debugger. Then when I actually do need a debugger, I have used it so infrequently that I forgot the commands, so I just use more Print statements.

Its a tool, like a yellow hammer; useful when you need it.

Since I write time-dependent C++ code I must test/debug the 'release' build. The debuggers universally screw up almost everything related to dynamic symbols (locals, parameters), so I have to read the assembler anyway to figure out what's where.

Naturally this makes it much quicker to use prints (logs actually) than struggle in the debugger.

I used debuggers long ago, but I've found that I work just as well, if not better, without them. I have to actually understand the code, not just get to a certain point and see what the variables are.

In the end, it means I know that section of code inside and out and can rewrite and/or refactor it as needed to solve the problem. It usually results in cleaner, more efficient code.

I agree with what several have already pointed out here, that a key point is that the famous programmers interviewed already understood their code and hence using a debugger to better understand it is not particularly useful.

I've been struck over the years at how one of key benefits of the Smalltalk environment -- being able to explore/change things in a program while it's running -- is not more widely regarded in the field. I occurs to me now that the interviewed programmers might have already gotten much of the cognitive benefit of such exploration at the beginning of a project. For them, print statements and listings are a way of confirming details of what they expect to be happening as the code runs. For un-godlike programmers (and maintenance work), debugging is complicated by the lack of deep understanding of how the code is supposed to work, hence the need for more exploration time and debuggers.

I used gdb plenty in C/C++ but that was mostly to debug mysterious errors when exceeding array bounds. Back then continously running server software that crashed would essentially move away its core file and restart, then just run gdb on it and mail be the output of a stack trace and "info locals".

In Python that's sufficient to catch most logic errors -- a full stack trace with values of all locals and member variables is enough to diagnose 90% of errors. When the error does not lead to an exception but undesired behaviour print debugging is easiest -- I have the software running so it restarts on source code changes and the restart takes a second and does not lose any state as it's a web app.

I could see myself using pdb if the Python software was a more complex server that took longer to restart and had some complex state that'd be lost on restarting.

I'm surprised to read more people arguing, "print statements and/or understanding the code I'm working on is more valuable than a debugger" rather than straightforward, "my platform/language/environment doesn't have a good debugger--print statements are simply easier." pdb is pretty good but wedging it into my FastCGI processes running as another user is a PITA (albeit one I could fix just once). Clojure has CDT but that's even kind of PITA-y since I have to go start up another JVM and get them to talk to each other. It's much easier just to type "print x, y, z" and reload my code.

If I was in VS.Net or Eclipse and not Emacs all day, and/or if I was writing something more "straightforward" like a desktop application, I bet I'd be much more inclined to use a debugger.

(Edit for word choice.)

I couldn't imagine coding C# without a debugger (Visual Studio).

I couldn't imagine coding Ruby or PHP with a debugger.

Maybe I've just not been incentivised enough to learn how to properly debug Ruby/PHP though.

FTA: "It saves me over and over again when dealing with code I don't understand. Not having it would be terrible."

Perhaps that is the problem. What are you doing working with, using, or writing code that you don't understand

To me this is a bit like the folks who randomly change their loop conditions until they get the desired behavior. Sure it works correctly now, but they have no idea why their changes made it that way.

My work is mostly in distributed real-time systems. All of the really hard debugging problems (e.g., race conditions) occur in situations where you can't pause one of the components with a debugger or you won't be able to repeat the problem. In those cases, logging events and studying them right after repeating the problem is your only recourse.

In 98% of my time i never use them (Java/IDEA IDE).

I use profiler a lot lately though (Java Visual VM).

A debugger loses much of its value in a functional language. Debugging Haskell or functional Scala code with a visual debugger is almost counterproductive.

I can't believe that someone would use a language that didn't have an IDE/interactive debugger. It's like going backwards in time to the early 80's. :-)

> ... Bertrand Russell ...

wat