Debugging Lisp: trace options, break on conditions | Hacker News Reader

Debugging Lisp: trace options, break on conditions

lisp-journey.gitlab.io

96 pointsakkartik29 comments

Related to debugging, the follow up article: how to fix and resume a program from any point in the stack. Have the debugger pop-up, read the error message, go to the erroneous line ("v"), fix the code, compile the function (C-c C-c), come back to the debugger, press "r" on any point in the stack, see the program succeed. You did not have to re-run everything from zero.

https://lisp-journey.gitlab.io/blog/debugging-lisp-fix-and-r...

https://www.youtube.com/watch?v=jBBS4FeY7XM

and some more: https://lispcookbook.github.io/cl-cookbook/debugging.html

I like Lisp but I've had one perpetual problem with debugging it that makes me feel like I'm doing something wrong. Once I've hit a breakpoint, and SLIME or whatever presents its options, HOW can I get into a REPL in the context of the stack frame, so I can eval expressions that reference variables in scope there? The best I can find is pressing `e` over the stack frame to evaluate one expression, but that's garbage compared to the full repl (and hotkeys that send expressions to it like C-c C-e).

I feel like there must be some way to do this that I just haven't found yet.

That's SLIME specific. SBCL's own REPL has it.

  * (defun foo (x)
      (declare (optimize (debug 3)))
      (let ((n 10))
        (break)
        (* x n)))
  FOO

  * (foo 3)   ; calling foo
  (foo 3)

We get a break repl:

  debugger invoked on a SIMPLE-CONDITION

  restarts (invokable by number or by possibly-abbreviated name):
    0: [CONTINUE] Return from BREAK.
    1: [ABORT   ] Exit debugger, returning to top level.

  (FOO 3)
     source: (BREAK)

Let's change the local variable N of the running function FOO:

  0] (setf n 12)
  (setf n 12)

  12

Return from the break:

  0] 0          ; restart 0
  0
  36

Note that you may want to reduce the optimization level of SBCL, since by default everything is compiled, and its optimizing compiler might get rid of a few of the variables during compilation. For example the compiler might detect that N's literal value 10 is not changed in the function and replace the variable access to N with the compile-time value. That's why I used the DEBUG quality 3 (highest) to tell the compiler to not optimize the code.

Fair enough, and I've done it just in SBCL like that before, but everyone always sings the praises of Lisp's (+ SLIME's) amazing debugging, and without the ability to stop at a certain stack frame and explore, I feel like I'm debugging with a blindfold on at all times.

one of the reasons SLIME is not my favorite dev environment

It actually can be done in SLIME's debugger but it's hard to discover especially if you don't guess that `h` is the correct key (I tried `?` first) to bring up a list of bindings. `e` will let you enter an expression in the emacs mini buffer to evaluate in the selected frame.

Superdisk's original comment mentions, and I agree, that pressing "e" on a frame to evaluate a single expression is bad UX. A relevant issue on GH* mentions that you can keep evaluating in the SLIME REPL, but I just tried and it doesn't seem to work poperly, the frame locals not being available in the REPL. Sly also doesn't seem to do it.

* https://github.com/slime/slime/issues/581

> We get a break repl:

How? I ran the same function, brought up the debugger, if I try to type something emacs says the buffer is read-only. Was I supposed to push e first?

SBCL's REPL. Outside GNU Emacs.

ah, got it! and was able to reproduce.

This will be something to keep in mind whenever trying out different REPL offerings outside of emacs / slime ... thank you for the tip!

I think you are doing it right, I don't know if it's possible to get a REPL in the context of the stack frame either. It would be handy and the workflow would resemble other ecosystems'.

I suspect the main blocker here is that it wouldn't know which "frame" to eval in. Since, from the debugger, you can eval in any frame on the stack?

If it's not done it would be a worthy addition.

This was a nice read! Thanks for sharing OP! Need more of these articles. Practical HOWTOs like this go much further along in convincing people to use Lisp than plain advocacy which more often than not rubs people the wrong away. If there are more articles like this that simply show what can be done and how it can be done there is not going to be a need for advocacy. A good HOWTO is worth a thousand words of advocacy!

I'm a very big fan of using sbcl's (sb-profile:profile function1 function2) (sb-profile:report) and (sb-profile:reset) and finally (sb-profile:unprofile)

For large production systems that may be running some functions millions to billions of times during a given process, the profiler is amazing for reporting how many times each function ran, how long each run took and the ranking from top to bottom of slowest to fastest enabling finding and fixing the slowest functions and achieving incredible application efficiency gains.

I wonder what it would take to bring these things to Clojure. Its REPL experience is miles ahead of non-lispy languages but I do feel a pang of grass-is-greener whenever I hear about CL's debugging tooling. Hell, the JVM has a great debugger as well (or so I hear), so why is that difficult to port over?

Same, I've been torn between learning Clojure—which I think is brilliantly designed, but I dislike its reliance on the JVM and exceptions vs conditions/restarts—and Common Lisp, that comes with a standard library that feels older than POSIX and C, but a REPL experience that is unrivalled and decades ahead any compiled language.

I decided to go with Common Lisp for its REPL and inside-out development experience, and just hope there's someone that will eventually take Clojure and CL, and fuse the two together.

The more I dive into Lisp and Smalltalk, the more I am convinced we are in the stone age of computing. We have reached the local maximum of UNIX and the basic abstractions over machine code we call compiled languages.

There's some good debugging tooling for Clojure as well. A recent entrant is https://github.com/jpmonettas/flow-storm-debugger and of course there's the estabilished pretty full featured debugging features in CIDER [1](Emacs), Calva [2] (VS Code) and Cursive (IntelliJ, using std Java JDI debugging). And for barebones tracing from REPL there's goo old clojure.tools.trace. And a bunch of others (sayid, postmortem, cljs-devtools for ClojureScript together with browser debugger, etc)

What CL has over Clojure is mostly the condition system I think.

[1] https://docs.cider.mx/cider/debugging/debugger.html

[2] https://calva.io/debugger/

I would love to know the answer to this! It sounds like something that would be both possible and very useful. I assume there's a good reason why it can't be done based on how the Clojure / JVM execution model works, but I don't know nearly enough about it to hazard a guess.

You can just use e.g. Intellij's debugger with Clojure? What exactly do you need ported?

Personally, I don't really see the value though. I prefer to create modules of mostly pure functions, testing them in the REPL using Rich comment blocks.

One thing is that when an unhandled exception occurs the clojure program is terminated, where as lisps usually does allow one to restart the program at the point of exception after examining what caused the exception and changing the values of local variables to rectify that. Can the intellij debugger do that?

Emacs, of course, can also do this for its own code with `trace-function-[background/foreground]' in addition to its usual `edebug' feature.

does emacs have conditional break and report ?

Edebug does have conditional breakpoints:

https://www.gnu.org/software/emacs/manual/html_node/elisp/Br...

Note that the extra arguments to trace are not defined in the standard, but are allowed by the standard. They are implementation specific.

You can also get something like arbitrary computation at function calls using :around methods.

Slightly OT, but when you edit code in the REPL this way using conditions/restarts, how do you avoid getting your code out of sync with the REPL's state? It seems with a long-running REPL, you eventually run into:

- Out of order execution means we don't know what order our code should be run in to achieve the current state

- If we run some code in the REPL and never save it in a file, then our state is also out of sync

- Finally, if we redefine some code, there's no way the state can be in sync with the code if started from the beginning

How do Common Lispers typically deal with these issues for a REPL that's been running for days or weeks, short of just saving the image and never turning it off?

As the cousin comment stated, we use sources. Even with this condition/restarts/debugger: fix the error in the source and recompile it, not in the REPL.

And we have the right to restart the image. For example, I run the tests from the terminal, hence from scratch, from time to time, typically before pushing my commits. I have a CI that tests and builds the program. I deploy a static build. We don't use images coming from development here. However, I can connect to a running image in prod and tweak settings if I want, or just look around[]. I could very easily change the code, but I'll do that in my sources and do a clean deploy. Or not. I too heard about people who mold a running image for years, their sources totally out of sync O_o

[]: there's a trading startup that posts screenshots of their Sly REPL from prod, that's where they ask their system for data. They didn't have to setup another complex layer just to see data from prod.

This is true of any image based development, as well. You can get the same thing in Java with hotswapping code, even.

That is to say, this is a bit of a concern, but isn't typically as large of one as you'd think. There are plenty of ways to make it so that you can't reason about a program. In general, you avoid doing those things.

Specifically to your question, I think, the biggest trick is that you rarely use the REPL as where you type your code. For that, you typically still use files and eval the file into the environment controlled by a repl. Even in emacs, you rarely just execute elisp from the scratch buffer. Unless you know it is something you don't care to keep, of course. Instead, you are working with files and evaluate the file on a regular basis.

repl should have a session back patch feature

whatever you typed, extract a patch and send it up to your editor for faster reconciliation

Reply on news.ycombinator.com