Show HN: Watchpoints, an easy to use watchpoints equivalent library for Python | Hacker News Reader

Show HN: Watchpoints, an easy to use watchpoints equivalent library for Python

github.com

  from watchpoints import watch

  a = []
  watch(a)
  a.append(1)  # Trigger
  a = {}  # Trigger

HOW?! I demand an explanation.

How does the second trigger fire? Deterministically, without relying on GC tricks? It’s watching a variable name, somehow.

Does it hook into globals()? But how would it know it needs to?

It uses Python's built-in inspect module to get the caller frame. https://github.com/gaogaotiantian/watchpoints/blob/68bc13716...

It then uses sys.settrace (which is intended as an interface for debuggers) to step through the code and check whether the variable has been changed. Documentation on sys.settrace: https://docs.python.org/3/library/sys.html#sys.settrace

Python exposes most of its guts as part of the standard library, making clever hacks like this possible.

Does this incur a performance penalty? What I mean is, could you theoretically write code this way with no ill effects?

Not that you’d want to, but it’s a fun thought experiment.

lol, thank you for the attention. Yes this will incur a performance penalty, and is similar to what "pdb" introduces(if your are familiar with pdb). Most of the python debugger did single step or breakpoint feature in this way.

And it's a little more than inspect and frames. Note that it is much easier to do watch("a") than watch(a), but it's slightly less intuitive. So I also did some AST hack to make watch(a) possible :)

Please make more stuff :)

Are you on Twitter or anything? I'd like to follow your work. (Put some info in your HN profile!)

I noticed that watchpoints doesn't work in the REPL, and since I live in the REPL, it limits my usage somewhat. But after digging into the code, I'm not quite sure if it even makes sense in the context of <stdin>...

I was also -- as the other commenter said, excuse the french -- fucking amazed that this works great:

    from watchpoints import watch

    a = []
    watch(a)
    a.append(1)  # Trigger
    a = {}  # Trigger

    def qq():
      global a
      a = 99 # Trigger


    qq()

Did not expect that.

I'm not on twitter, but I do have a twitter account for my other project viztracer @viztracer. All my work is on my github if you are interested :)

watchpoints does not work in the REPL. REPL is a more complicated environment. I don't think pdb will work in REPL either.

watchpoints works very intuitively. It has some dirty magic that made it work in some way that you need but not expect.

This is very cool - thanks for sharing.

I've been working on a tool for automatically suggesting performance fixes to python apps based on static analysis and dynamic profiling. This is a nice method adding additional dynamic information that you need for certain optimizations. (E.g. if you want to suggest replacing a list with a set then you need to know how that list is used which sometimes can be hard with static analysis.)

Very interested in trying this out! And good timing for me personally as after a decade of using SublimeText, I tried VSCode because I was looking for a better was to debug python code. This also looks like a better way to debug!

Thanks to make great lib! Is it expected to use with pdb or console? I thought it might be useful to trace with logger.

By default, you'll write this debug code piece in your program and it will print to console. However, you can design your own callback so it can work whatever way you like.

Also I already implemented the pdb hook so this could work like breakpoint() in built-in library.

Pardon the French, this is cool as fuck! I'm gonna use this in literally everything ever.

It would be nice to have this integrated into pdb/pdb++/ipdb

This looks very useful. Thanks!

this is great, python debug tooling seems to be getting a lot better recently.

This is great!

Reply on news.ycombinator.com