“Give me 15 minutes and I'll change your view of GDB” [video]
undo.io
undo.io
1. introduction to 'tui' mode in gdb (https://sourceware.org/gdb/onlinedocs/gdb/TUI.html)
2. intro to python support in gdb (if it was compiled in)
2.1 python print('hello world')
2.1 set breakpoints with python gdb interface
Aaron gave a great talk about his switch here: https://www.youtube.com/watch?v=JWD1Fpdd4Pc , which caused several people at work to move off NeoVim and give emacs a go.
In the old days, Emacs alongside DDD was the only way for me to get some of the Borland IDE comfort in UNIX land.
"watch p_mydata[34]->blahblah" sets a hardware watchpoint;
Ctrl-x-a-b sets a breakpoint at the cursor in the Emacs;
Ctrl-x-a-u runs until the cursor;
Ctrl-x-a-j jumps at the cursor;
And many other niceties.
There's also a way to create a buffer to monitor some variables.
You can actually attach the emacs-gdb buffer to a process you're running through valgrind when a valgrind error occurs. Not to mention it runs undo-gdb, of course.
How come the open source world hasn't been able to create a debugger that's as good as the (still primitive) debugger in Visual Studio 6.0? It's astonishing to me how bad the state of the art is.
These tend to have zero actual content (emacs is a text editor. How would I use it with gdb?), and not provide any evidence, or guidance.
I know you could put watch expressions into gdb, and I suspect there was a way to pull these into another emacs text window, as well.
Gdb also had commands to list the stack and select a frame further down to see the callers data. I don't remember if the source code view (in emacs) would track the selected frame.
I keep asking myself that same question every now and then, why can XCode have such a nice and pleasant LLVM-based version of gdb, while gdb on linux still doesn't have even the most bare-bones GUI that let's me inspect common data types without jumping through hoops?
you mean lldb?
That said, there is great free software with good UI/UX too - only they are more or less always backed by a large corporation.
It's not perfect- there are still rare situations in which it's not using the right symbols, but at least that I can usually fix that with a 'target symbols add'. Otherwise it's because the thing I'm trying to debug is sandboxed or otherwise protected, which is almost always going to be a hassle, and requires workarounds outside of the debugger. Raw lldb is a lot better than it used to be though.
Very frustrating when it crashes just as you've got the target into the state where you think your bug will trigger. And then GDB repeats the performance reliably for you... On those days you end up resorting to a lot of creative thinking.
There are plenty of nice front ends for gdb on all platforms though, including Linux: https://sourceware.org/gdb/wiki/GDB%20Front%20Ends
When I was into Linux in the '90s, DDD was very popular (as was Emacs as a gdb front-end), but now there are also dedicated IDEs like Develop/QT Creator/Eclipse that provide similar front ends to gdb like Xcode does for lldb (and used to do for gdb).
It's also very valuable to learn the CLI (both for gdb and lldb), because it seems like there are inevitably useful features in both debuggers that aren't exposed by any GUI, not to mention being able to debug something over ssh. It's not a requirement though, there are tons of options out there.
I'm a bit skeptical. Like the rest of LLVM, it's probably "fundamentally" an API, but with a bundled command-line interface.
could you elaborate on what's missing in GDB? data-types are written in the binary by the compiler (dwarf format), and GDB does nothing more / nothing less than parsing it and giving access to it through it user interface.
GDB has no knowledge about your code or it's library in itself (at least in C, Python extentions now include libc++ data-type-specific pretty-printers)
I rarely bother; it's easier to fprintf() and read the log file, and this has the advantage of also working when you're trying to debug asynchronous processes.
Because nobody has hired developers to work on it. Most large scale projects (whether OSS or Proprietary) get done when someone bankrolls the development. And I suppose the reason nobody has hired developers is because its hard to make money on open source debuggers.
I still wonder what's missing in GDB. I got quite proficient with this tool, and never had access to VS. I personnally don't count GUIs built on top of GDB, as they don't provide new capabilities, just 'fancier' interfaces that GDB's CLI.
A strong visualisation and comfortable interface is not a new capability per se, but still a very important (some would even say "killer") feature.
Unfortunately, we're stuck with gdb, and improving it or making a new debugger requires massive effort and likely backing by one or more organizations (e.g. lldb has apple).
The ugliness of gdb won't change, but this presentation isn't about gdb's shortcomings -- it's about a "version" of gdb that supports reverse-debugging, which is really awesome.
I use undodb-gdb all the time and it's amazing. Consequently, I use gdb all the time. The result is that over time I've learned to deal with gdb's ugliness and so I get used to it. It's similar to complicated or hard-to-discover user interfaces in GUI programs.
What about rr?
It's built on top of gdb, but seems to have a straightforward workflow (at first glance at least, I've not used it yet).
How so? I saw this suggested in some places but I found windbg atrociously complicated, even compared to plain GDB. It can debug the windows kernel, but that's about the best I can say about it. x64dbg and IDA's debugger were far nicer to work with for all desktop reverse engineering purposes.
In terms of non-RE debuggers, yeah, VS's is pretty good, but I don't find it that much better than anything else to say it's a must-have. Most source-level debuggers offer about the same feature set in only slightly different ways. It does the job and once you know one you pretty much know them all.
Seeing what a single person (Greg Clayton) has done with his gui experiment for lldb (just run lldb and type "gui" command), I'm not sure the effort is so massive. Rather it's the fact that OSS developers have a really weak spot when it comes to design a good UI.
Also, there's Voltron:
-> Eclipse can be easily be started from the command line - e.g. "cdtdebug.sh -e ./a.out". A minimal version of Eclipse opens up, and you can immediately step through code, no need to create projects.
-> While GDB TUI mode somewhat mitigates the issue of not having context when debugging it IMHO pales in comparison to seeing the full file in Eclipse.
-> In Eclipse I can see so much more information than gdb. I can see several expressions all being updated, memory views, I can highlight expressions in code. With gdb I could only ever see one thing at a time.
-> Although GDB has more functionality than Eclipse, I found since I couldn't remember certain GDB commands and so I just didn't use certain features that I find myself constantly using in Eclipse because it's just a click away.
20 years later apparently it's still maintained, even if it has plataeued.
Personally, I'd like to see a more modern frontend made, perhaps for Atom.
A debugger is something that helps sell an IDE by padding the feature list more than something most people will use much.
I'm not saying it ought to be like that, but in more than two decades of watching developers work, my conclusion is that most developers 1) are unfamiliar with debuggers, 2) if they do know about debuggers they rarely use them, 3) if they do use them, they rarely use more than the very basics .
I myself fall in category 3. I use gdb, and I'm fine with that because all I tend to use is going up and down the stack, displaying some expressions and occasionally single-stepping.
I've used Visual Studio, and I've used DDD, and frankly I fall back to the above because it's quick, and because it's very rare that spending lots of time in the debugger is as helpful as writing more test cases or spending more time reasoning about the code.
To displace more primitive debugging techniques, a debugger will need to be extremely quick to cut to the core of the problem and basically help automating what is often a very basic workflow to try to narrow down things like "that crash happened because of an incorrect value in x; when did x get that value and what contributed to that?" followed by "how did y get that value that contributed to the broken value in x".
You can provide lots of fancy ways of making those kind of steps more pleasant, but they're doing too little to beat having a very minimal UI to deal with.
Instead debuggers appears to largely have focused on making those hair-pulling frustrating and long but also exceedingly rare debugging sessions more pleasant.
At least that's my impression: 99.9% of the time, I need something that's a smidgeon above a printf; 0.1% of the time I get frustrated enough to consider looking at more advanced tools, but that's not often enough to usually be worth it.
Because you have the most experience in environments where debuggers can only reach a small fraction of their potential. (Barely better than printf) Let me assure you that there are very different environments out there, and have been for decades.
(What I'd like to see is more context aware automation of the environment; e.g. debugging a crash? I'd like to see attempts at tracing values backwards, and options for attempting to re-try runs with automatically added probes based on the state of the environment when it crashed; done right, that's save me time)
Java.
For all its warts Java has a pretty decent open source community and many of the best tools in Java land are open source.
For example all the three major IDEs are available in open source versions, two of them are completely open source (although you can get commercial plugins) and all have decent debuggers.
It's still kind of crappy, but at least it's not ed.
I know this is really-really hard to accomplish, but some IDEs do it.
The only thing that's "optional" and a little over the top (but very useful) is time-travel debugging.
You might make the same argument against unit tests, or little hacks you write to separate out a problematic piece of code from an even more complicated context.
Live recompilation in Java (at least in Eclipse) behaves basically the same way.
It occurs to me now that's essentially using the debugger as a REPL, but with access to a whole runtime's worth of external state. That's not a bad tool to have in your box.
The question at the core here is: is a debugger meant for experimentation to find the (often subtle) root cause of a defect, or is it meant to be a read-only window onto state during execution?
Best case thought, it's so useful. It's also incredible for editing tunables on the fly(think videogame gameplay values, layout values for UI, etc).
Oh sure... VB6 and good OO design and modern practices (unit tests, dependency injection, etc) don't mix. But... if you need to sit down and "hack out" a bunch of working code quickly it was pretty hard to beat.
Python with the VB6 ide/debugger experience would likely take the scientific computing field by storm.
I miss it... especially since I've never been able to get "edit & continue" to work at all in Visual Studio.
When I last wrote some VB6, I was criticised by a colleague for 'over-engineering' because, to build a cancellable progress dialog, I used a (very simple) observer pattern. He didn't understand what it was or how it worked - and would have preferred to simply block the entire UI while a process ran (without the ability to cancel, even). To him, even basic OO ideas were pointless and dangerous over-'designing'.
If your application was crap, it could also remain incredibly simple in terms of design, though inevitably multi-thousand line methods and hacks upon hacks led to completely non-understandable, buggy, and brittle code, so I'm sticking to my 'enough design' principles!
One more aside: This developer complained that there was a bug in the VB6 IDE because it wouldn't let him add any more code to a file - he'd written such a huge code file that he'd actually hit the IDE's limit. I tried talking about modularisation, refactoring, etc... then just gave up.
Do Smalltalk workspaces do something fundamentally different?
When I tried out pharo I was pressing backspace all the time because outputs are not valid code and thus would break the highlighting.
Anyway, a repl is just that, a read eval print loop. If you interface with it using a command line or other means is just an implementation detail.
If you're in a codebase composed largely of side-effect free functions or well encapsulated Object Oriented code, it's a very good way to debug. I had great success with such debugging and even coding and new development in Smalltalk environments for over a decade.
On the other hand, if your codebase is full of side effects and doesn't have good encapsulation (perhaps there's a lot of fiddling with globals) then you're going to have a bad time. But to me this isn't because the debugging method is bad. To me, it's because your codebase is designed with lots of tight coupling and side effects. You have an architecture that makes it harder to reason, debug, and refactor your code. This isn't just spouting. I'm basing this on many years of experience. And yes, I saw both kinds of Smalltalk code, and the effect is exactly as I described. Guess which codebases were more productive?
I suspect your sample is a bit skewed.
The Smalltalk class library wasn't stateless. It certainly wasn't pure. Really, code only has to be "pretty good" for such techniques to work well.
There is all of this excitement around ideas like Light Table, allowing you to see the output of your code inline as you develop it. Being able to alter code and recompile on the fly in the debugger gives you a very similar experience.
IBM also had a version of C Set++ that used a Smalltalk like image database for C++ code, but they hardly managed to sell it as it was very resource hungry for hardware at the time.
It is very hard to find online references to it though, I think I read about it on Dr. Dobbs, back in the day.
A video from Energize C++ for those that never saw it
But as that person in the audience says, it looks like the presenter is just not aware that the GDB TUI is a substandard version of Emacs GUD, having mostly the same keybindings.
The second half is a good combined demo of some neat GDB features though.
https://lists.gnu.org/archive/html/emacs-devel/2015-02/msg00...
If LLDB eventually becomes as popular as GDB, I'm sure it will be included into core Emacs then. But I have no interest in LLDB at the moment.
No, actually, it won't :) RMS has said it won't, in fact.
And even if he was, I don't believe he ever said "If LLDB successfully replaces GDB as the most used libre debugger, even then we still shouldn't support it in Emacs". He's not insane, you know.
That said, the debugging I do is all from the command-line anyway, so all I really want is a way to quickly view the assembly and registers when I break, which the TUI does quite well.
As a note - I think the person in the audience was trying to get at the fact that gdb (like tons of cli programs) uses readline() for input. readline() can be configured to use emacs or vi keybindings, with emacs the default - hence why ctrl-p and ctrl-n work. I personally use vi keybindings, so in gdb I can use ESC, and then jk to go up and down the command history.
GDB is a fine debugger. I've never had any trouble with it at all, and I don't recall ever needing a feature in it that it didn't support but other debuggers did, although I'm not a very demanding user of the debugger; I rarely need more than some conditional breakpoints and the ability to see the value of any given variable or memory location.
I'm fine with gdb and conditional breakpoints is usually the most advanced feature I need. Only once I used bigger guns. Detecting the error meant run into conditional breakpoint A, then enable and run into conditional breakpoint B (which would have triggered a lot before A), and then inspect a variable. The problem: The error only occured sometimes. Thus automate this via Python, so gdb could auto-reexecute everything until it found the failure case again.
What I still miss is multi-process debugging. Gdb can only attach to a single process, so you need multiple gdbs to watch multiple processes and this introduces race conditions.
Bring it, WinDbg :)
> people conflating the debugger with an interface to the
> debugger?
The reverse debugging stuff is pretty cool though.
I'm pretty sure if you asked anyone that did PS3/PS4 development they'll probably say ProDG.
Compared to VS and PIX it was night and day, although you probably already knew that from my comment.
I also use it for debugging unit tests for a static binary translator. When I'm unit testing, I will set a brakepoint at the reporting of a failure, look at how the native program and emulated program states diverged, then I'll use reverse execution and data/code breakpoints to go back and forth to try to figure out what went wrong. This is much better than having to restart each time.
Over the years I've found many bugs in UndoDB and they've been pretty quick to fix them. Their support is great.
I started using it as a student, on a non-commerical (i.e. free license), and it was an essential purchase for my current work.
However, this is a novice trap, as more experienced developers would know that printf debugging may not work for subtle cases, and may even steer you in the completely wrong direction. But by the time said novices encounter such bugs, they've already picked up the "bad habit", and it's difficult to go back and unlearn it.
Some also feels that needing a specialized tool to find bugs is a weakness, as you ought to be able to reason about the code without aid. But that is folly, for if so, why use editors with syntax coloring (shouldn't you know int is a keyword w/o being pointed out) or auto-completion (shouldn't you memorize all exported functions)? In fact it's the opposite: a good debugger greatly aids in comprehension, since you get to catch corner cases happen in action, and with sufficient context for you to figure out how it'd happened.
There was a time when many developers also don't use a D(VCS) of some sort, for similar reasons. But of course git has a celebrity champion, and so the rest followed. Almost all the excuses from this thread people had raised about not wanting to learn how to use a debugger properly apply to git (and to all D(VCS) to varying extents), but you'd be ridiculed for not using it. IMO debuggers need a similar champion before our opinions change, but alas IIRC Linus himself is not too fond of debuggers...
That said, the state OSS debuggers is terrible, so they're not helping their own cause. Moreover, they are often "buggy" themselves. I put that in quotes because sometimes a "bug" is actually correct, but unintuitive behavior for inexperienced users (pop quiz: how do breakpoints work?), or just poor / nonexistent documentation. Nonetheless, the result is the same, and nothing loses your trust in a tool faster than finding bugs in it while trying to fix your own bugs.
However, mentioning advanced-gdb usage nowadays and not mentioning RR (http://rr-project.org/) is a sin =)
Missing from this video, but the most powerful part of using gdb for me recently has been the ability to do procedural debugging. You can easily write debug scripts that are re-runnable instead of reproducing bugs manually. You can sequence together multiple conditional breakpoints, keep debug variables & state, and other stuff that is really tricky to do in more common GUI based debuggers. I haven't used Visual Studio in the last few years, so maybe they've added it. I had tried for years and never found it, resorted to gdb for the hard problems, when I could.
The debuggers I have encountered seem a bit limited when regarding the debugger as a means of querying a subset of program executions (out of all possible executions), but they are usually helpful in picturing system dynamics. It has always been very helpful for me to allocate a portion of fixed screen real estate to memory, variables and registers that I am watching and see the changes as I step through, allowing me to observe the dynamics of the system.
Too bad I could not figure out a way for gdb to help with that, the closest I got was examining the memory I was interested in ('x' variants) before and after, but no highlighting. To be fair, it is good at other things though :)...
This criticism applies to the UI overall, which puts things in inconvenient places (the tall, narrow find results/compile results panel was particularly ridiculous), and then won't even let you move them, but I found it especially tiresome when debugging. This is the last thing you want in a debugger UI in my view! I want to see as much stuff at once as possible, and I don't want to have to switch between tabs unless I deliberately set things up that way myself.
There's no separate registers window (it shares its window with the locals), there's only one watch window, and the disassembly window never seemed to work very well (when stepping, execution still proceeds on a line by line basis). I recall the disassembly window also sharing a window with the source window too. I think the memory view might do that too. All in all, most unsatisfactory.
Visual Studio does have some odd limitations and annoyances, but you can at least put the windows where you like.
https://blogs.msdn.microsoft.com/vcblog/2015/11/18/announcin...
Maybe polishing that plugin could go a long way in a better dev experience than the current GDB UI?
With large programs, there can be a really long distance between where something is created/modified and where it's used. If there's a bug you catch that occurs where the object is used, reversible debuggers help you trace back to where it's created, or who modified it last.
If you're at a place that's hard to put a useful breakpoint (e.g. a widely used function), and need to answer a what-if question, being able to manipulate state from the debugger is super useful. You might not be able to easily insert code to the program to create that state without messing up everything else (again, example is a widely used function, where inserting code could mess up other users of the function).
One day maybe we'll see a good non-buggy GUI frontend for GDB that shows code, local variables, assembly, etc in separate windows, all clickable to change values, set breakpoints, change interpretations, etc.
Not anymore! My, how times have changed.
I'm curious what people use the Python interpreter for?
This guy is my hero.