Sys.monitoring: Python Execution event monitoring | Hacker News Reader

Sys.monitoring: Python Execution event monitoring

docs.python.org

67 pointsingve9 comments

Does anyone know how this would be used in practice? An example would be useful because words like "tool" and "event" in this context are too abstract to make it clear how this is intended to be used.

One example is reimplementing coverage.py with this, as its made to reduce overhead for the types of tools enumerated in the docs.

https://nedbatchelder.com/blog/202312/coveragepy_with_sysmon...

One example I thought of is to use the ability to tap into exceptions being handled to audit for places where performance could be improved by swapping exceptions for sentinel values.

Iirc there's a significant performance penalty for raising exceptions. So you could tap into exceptions being handled to see something like "requests that fail validation raise an exception, and this happens often enough that we could get a performance boost by returning a sentinel value instead".

Another idea is that you might be able to use the conditional hooks to write a tool that can suggest re-ordering your if-elif-elif-else kind of statements so that the most common path is first, and prevent evaluating the other possibilities more often.

At a high level, it's just machinery for getting metadata about your code's runtime as part of the interpreter. You could probably write a lot of this as decorators, but because this is part of the interpreter it's decoupled from your frameworks/libraries/etc.

Well, I'm working on reimplementing the pydevd debugger to use it.

The general idea is that pydevd will be able to use that API instead of relying on sys.settrace (which was perceived as slow in general -- pydevd got by because it had a bunch of tricks to just trace the needed contexts but implementing a fast debugger in Python with it is pretty hard).

My initial results are still mixed -- i.e.: on some cases it's definitely faster -- such as when tracking exceptions, but at this point in all other scenarios it's still slower (it's pending a few profiling sessions and I already have some ideas on where to improve), but I'm still not sure it'll ever be as fast as the version that can hook into the python frame eval and change the bytecode for the function to add programatic breakpoints... time will tell (but that approach is also very hard to keep up to date on new python releases, so, I'll probably end up deprecating it as I don't have enough time/resources to keep it up to date).

Anyways, I have most tests already passing, but I have to do a few profiling sessions before the initial release. I guess there's no much point in saying: here's a new version of the debugger using sys.monitoring -- does the same but is slower ;P

Yeah - it's a bit opaque:

> Note free_tool_id() will not disable global or local events associated with tool_id, nor will it unregister any callback functions. This function is only intended to be used to notify the VM that the particular tool_id is no longer in use.

Why won't it do those things? Why would I want to notify the VM of this without unregistering?

Neat, I had no idea this had made it into 3.12. Looks like a nice, modern alternative to `sys.settrace` and its ilk[1].

[1]: https://docs.python.org/3/library/sys.html#sys.settrace

It’s funny how this bit of code was clearly written by systems engineers. Tools 0-5, bitmaps, etc. If this was another team you could have unlimited tools, IDs would be UUIDs, and freeing a tool ID would take care of all your loose ends like still registered callbacks.

Wow. Just think of the logging this could allow! And with AI … I’m getting ideas.

sys.monitoring is a legendary feature for python Recently, I came across a profiler based on sys.monitoring.

akarsu(https://github.com/furkanonder/akarsu) is new generation profiler based on sys.monitoring. You must try guys!

Reply on news.ycombinator.com