Pyringe: Debugger capable of attaching to and injecting into Python processes | Hacker News Reader

Pyringe: Debugger capable of attaching to and injecting into Python processes

github.com

120 pointscodecondo30 comments

Note that there are already a few similar projects:

* https://github.com/albertz/pydbattach (my own :))

* PyInjector (don't find a good link right now)

* I'm quite sure I have seen at least one other similar project here on HN a while ago, but I don't remember the name...

Nice list of alternatives. Good job you managed to find a link to your own.

Don't be a dick.

This is probably the most featured / well known one:

https://github.com/lmacken/pyrasite

We've been able to do this in C at runtime for decades. Just use strcpy.

Humour aside, in 10-15 years time I expect it to be considered strange not to have hot-swap features in production languages.

XCode used to be able to do this with C and ObjC code, but they dropped the feature when they moved to llvm.

That's interesting. Why did they drop it? Because of lacking support in LLVM? Are they planning to reimplement it? Or because of stability issues?

The immediate reason was lack of support in LLVM, but I suspect the reason why it wasn't re-implemented was that the demand wasn't enough to justify the significant amount of work that would be needed to get it working a) with LLVM and b) in the locked-down environment of iOS.

The speed boost from LLVM, SSDs, the "thin" nature of iOS apps, and the bump in processor performance was, per my intuition, enough that a typical "full" build today can handily beat a typical "Fix-and-Continue" build from a few years ago (I use the term "full" to mean a build that updates one or two compilation units and re-links).

It was a nifty feature and I wish they'd bring it back, but I'm pretty sure the XCode team has lower-hanging fruit (static analysis, code navigation, refactoring).

Just to note: Pyringe is not an official google project (we tend to also include googler hobby projects in the org). It's an intern's project that he wanted to open source.

So it's not "Google's Pyringe".

You're absolutely right, and I'm a colossal idiot for not making that absolutely clear.

You aren't an idiot, it happens :) Just don't want people to get confused.

The title is a bit misleading, since Pyringe is "only" able to inject Python code into Python processes. Aside of that, I think this could prove useful for Clojure-like live coding.

Looks more like being able to get a meta-REPL going in a running process...as could be done with LISP and Smalltalk since the 70s.

Shameless plug: my company develops Deviare[1]: "code interception for dummies". It allows to intercept a Windows application from any COM capable programming language.

We have interesting code samples such as capturing video intercepting DirectX[2] and hooking into SQL Server to abort unsecure (e.g. injection attacks) queries[3].

1. http://www.nektra.com/products/deviare-api-hook-windows/devi...

2. http://blog.nektra.com/main/2013/07/23/instrumenting-direct3...

3. http://blog.nektra.com/main/2013/06/26/sql-server-intercepti...

Do you know of anything similar for amd64 Mac/Linux? Not PLT hooking but the sort of deal where the beginning of the function is overwritten with a jump to the hook. The overwritten instructions in the original function are copied to a mmap()ed area of memory that jumps back past the hook jump.

I found an old copy of MobileSubstrate that has partial support for i386 and x86_64 (which works fine on Linux since the ABI is the same as Mac). I found out that it doesn't support all the possible jumps in the overwritten area. I've been slowly patching and improving it[0] but I'd rather be spending my time hooking stuff than writing the hooker.

[0]: https://github.com/jevinskie/substrate

I forwarded your comment to the right people in my company.

In the meantime have you tried to ask these kind of questions in http://reverseengineering.stackexchange.com/

https://github.com/rentzsch/mach_override

https://github.com/rentzsch/mach_inject

Thanks. I actually had those repros starred on Github but I must have forgotten about it. I'll look into porting mach_override to Linux.

You're welcome :)

What's wrong with http://pyrasite.com/ ?

Depending on what state cpython is in when you inject, pyrasite may deadlock or segfault the interpreter. All it does is try to get the GIL and then inject the code.

Pyringe first gets to a safe place before injecting by (essentially) faking the delivery of a signal, then waiting for python-level signal handler dispatch to trigger. By doing so, it ensures the interpreter first gets itself into a state in which it can execute arbitrary python code before performing the injection.

Pyrasite has a bunch of features pyringe doesn't, though.

Using a signal may also have side effects on your program.

It doesn't really send a signal to your program. It just sets flags inside cpython to make it think so. (pendingbusy and pendingcalls_to_do, to be precise)

why not just contribute to pyrasite to have this different method of injection instead of building out a new project? what is the tradeoff here?

This is super cool. With C programs, you can already attach gdb to a running program and start debugging, without starting the program under the debugger. Before pyringe, you couldn't do the same with Python. This will be very helpful for me at work.

You could always do this with any process no matter what language it's written in. The issue is in support for debugging at a higher level.

Yes, sure. GDB-level debugging of Python programs, without something like Pyringe, is pretty useless. So I am excited for the Python-level tool for debugging Python.

With gdb you have "py-bt" etc: https://docs.python.org/devguide/gdb.html

Note that you can call PyFrame_FastToLocals if you want to access all locals. This will selectively disable fast locals on a frame.

What I'd love to see is a version of this tool, which lets you interact with binaries, in Python:

http://duschkumpane.org/index.php/mmbbq

It's not what is posted here, just what I thought it was going to be.

Reply on news.ycombinator.com