Checking Out CPython 3.14's remote debugging protocol
rtpg.co
rtpg.co
- It'll be possible to print stack traces without modifying or stopping the program.
- It'll be possible to exec into a program at runtime without modifying it.
I'm not sure why the author mentions remote_pdb - this has been with Python for some time, and works since Py 2.7? Not sure what changes in 3.14 for remote_pdb.
What I'm hoping though is improved tooling around debugging Python. Currently, in my experience, VSCode (more specifically, debugpy) provides pretty much unmatched remote debugging capabilities, and I'm really hoping we can have a standardized way to connect any IDE to remote Python processes with the same UX as VSCode.
I would love to use something like Zed, but without remote debugging abilities, the IDE is pretty useless for me. Perhaps better devs don't need remote debugging, but I depend on it more than a junior in college CS program depends on AI :)
This is what 3.14's remote debugging protocol gives us. And even more than that, because I believe that debugpy needs to be invoked beforehand!
The value of this tooling is you don't need to predict that you need debugging before running the program. All of your Python programs[0] can be debugged. No need to restart. No need to modify the launch parameters either: you don't need the "debug configuration" from vscode, you just need a PID.
[0]: so long as you're using the same version of Python and it has the remote debugging protocol enabled, and also you have the right OS permissions
[0] https://pyrasite.readthedocs.io/en/latest/ [1] https://github.com/prompt-toolkit/ptpython
I'm working on a live coding environment for python[0], based on emacs' SLIME mode for common lisp. It's quite new and I haven't written documentation yet, but all the main SLIME features not covered by LSP are working.
- All results printed in the repl are presentations that can be inspected, copied around and used again -- as the actual object, not just it's str or repr text like in most repls.
- On any uncaught exception you get an interactive backtrace buffer where you can jump to source, see arguments and local variables for each frame, and eval code or open a repl in the context of any stack frame. And the arguments and local variables aren't just text but presentations you can open in the object inspector, copy to the repl and use, etc.
- A thread viewer where you can view stats on all threads, get the backtrace of any thread, spawn a repl in the context of any of it's stack frames, etc.
- An async task viewer with somewhat more limited functionality as async tasks don't keep a full stack.
- A pretty documentation browser using mmontone's slime-doc-contribs.
- The ability to trace functions, where again their arguments and return values aren't just printed as text, but as presentations, that you can open in the inspector, copy to the repl, etc.
- I took some code from IPython's autoreload extension, so interactive development without restarting and losing state mostly works.
If you want to collaborate or just talk ideas that'd be fantastic, I don't have any experience with the Pharo/Smalltalk world.
I really like debugging in a simple shell (à la gdb) so this would really be nice for my workflow.
I've found that the best way to inspect Python programs with GDB is to use the GDB extensions (https://github.com/python/cpython/blob/3.13/Tools/gdb/libpyt...) that the CPython project maintains. It's a bunch of code that tells GDB how to navigate the interpreter internals and how to find things like the local variables and the backtrace of the currently executing function. You can actually see quite a lot using this! It falls short if you need to do any kind of navigation or stepping, but it's great for understanding what your program is doing "right now".
At my employer https://undo.io we makes a Time-Travel debugger called UDB which is similar to rr and we've had some success introducing time-travel into this Python debugging approach. We can record and replay Python processes, and with some additions on top of the GDB extensions from CPython we can navigate around the execution history. It's a bit rough around the edges, but it's really powerful to be able to evaluate Python expressions at arbitrary points in time, and we're finding it valuable to use internally. Here are some more details if you're interested (you should be able to adapt this to rr): https://undo.io/resources/how-i-debug-python-code-with-a-tim....
I expect that this new remote debugging protocol will be useful to some of the approaches we're currently testing out.
If you want to "just" use sys.remote_exec, you will need to use that from a matching Python version. If you dive into the implementation (check out the PEP), you could most definitely write a C or Rust program that implements remote_exec and "does the thing".
The protocol is basically "tell the program to read a python script from this file". That python script will be executed in the running program's environment, but you don't need for the software on the other side to be a python program!
sys.remote_exec itself only targets same Python versions though.
Which brings me back to my long-held belief that python functions are not quite first-class, not until you can rip them apart and put them back together as if nothing ever happened.
Not to say there haven't been improvements, as evinced by the decorator module rewrite from version 2->3 (or 3->4; can't remember) that completely changed how the resulting function was constructed.