Show HN: Wat – Deep inspection of Python objects | Hacker News Reader

Show HN: Wat – Deep inspection of Python objects

github.com

393 pointsigrek5192 comments

Waaat :D. This is so nice. I used to use python-ls[0] for similar, but something about it that I can't recall broke for me and it's no longer maintained. Adding to my debugging arsenal which primarily consists of snoop[1] and pdbpp. Only thing I'd like for wat now is maybe an ipy widget for even easier object exploration in Jupyter.

I'm also really appreciating the base64 exec hack. All my years in Python I never thought of or came across it until now. I'll totally be using it for some things :).

[0] https://github.com/gabrielcnr/python-ls [1] https://pypi.org/project/snoop/

For object inspection in Jupyter if doing more numpy type things, check out penzai: https://github.com/google-deepmind/penzai

Seems like if you add more "a"s to W[a..]t you slide a scale from disgusted to awed to comical.

Ah this looks fun! I use "dir" all the time with python, and find it more useful than official documentation in some cases where documentation is not great.

Surprised there isn't more innovation and new tools like this around python's interactive shell given it's one of the real strong points the language has.

There’s the help() function as well! Super helpful!

And if you use IPython, you can use the ? shorthand, too:

    In [1]: int?
    Init signature: int(self, /, *args, **kwargs)
    Docstring:
    int([x]) -> integer
    int(x, base=10) -> integer

    Convert a number or string to an integer, or return 0 if no arguments
    are given.  If x is a number, return x.__int__().  For floating point
    numbers, this truncates towards zero.

    [...]

the double questionmark `foo??` is sometimes also useful, since it'll show full source for the function/method/thing if you do it to a function.

Favorite way to develop python, so far:

1. Write the program skeleton

2. Put an import pdb; pdb.set_trace() whereever I need to fill out next

3. Once I hit the breakpoint, type 'interact'

4. Start coding in the debugger.

4a. For small functions write in a scratch pad, test against another python console, then paste into the current debugger.

4b. For network calls, save the inputs and outputs. Toss a decorator on some functions to log the inputs and outputs to files, and cool we already have some proto test data.

4c. For documentation, call help() on the variable or method or module.

Looks like a fancy version of good old icecream. https://github.com/gruns/icecream

If you never heard about it, scroll to the bottom to https://github.com/gruns/icecream#icecream-in-other-language...

Neat. I built something similar but web-based for Java years ago: https://scg.unibe.ch/wiki/projects/DoodleDebug

These kinds of tools are useful.

Twenty years ago I wrote an object introspector for Zope ;)

Nowadays, I'm using devtools daily, and icecream and q occasionally. I'll give wat a try.

> from wat import wat

Given the cool nature of this project, I'm surprised they don't offer simply "import wat" with identical usage syntax. Thus inviting curious users to wat/wat in order to discover the trick...

"import wat" would be great, but Python has some restrictions about modules not being callable. That's why I ended up with longer `from wat import wat`. Not sure but maybe this would be more convenient `import wat; wat.wat / object`

Python has code execution at import time. Just grab the global context, and overwrite the module with a callable.

Afaik the import sets the module in the importing module's context only after the code in the imported module is run.

Edit: Oh, you don't mean in the importing module's globals, you mean `sys.modules`. Yeah that works!

actually, no you were right about what I meant but i'll claim credit for helping you realize there was a better way XD

Check out the module “q”! It’s callable, its author talked about how great it would be to have a module level __call__ because the way it was made callable is super wonky.

> the way it was made callable is super wonky.

why do you say it's wonky? it's just

    sys.modules[__name__] = wat

According to the official python documentation [0] of sys.modules:

> This is a dictionary that maps module names to modules which have already been loaded. This can be manipulated to force reloading of modules and other tricks.

Thus, the modules dictionary is being used as intended, and it has the desired effect. Nothing wonky about that.

Of course, a __call__ method would be better because you could still keep the other functions inside the module. But for a single-function import like "wat" it seems quite natural and sane.

[0] https://docs.python.org/fr/3/library/sys.html

look at what q does: https://github.com/zestyping/q/blob/cbad74095e20017f33d3ab75...

I wrote this a long while ago: https://pypi.org/project/funkify/

See snoop[0], which does just this. It also has an install function that registers it in builtins, making it universally available.

[0] https://pypi.org/project/snoop/

Thank you all, I learned from you that it's possible to have a callable module. Although `sys.modules[__name__] = wat` looks like a black magic and I'm a bit afraid of locking out to other importable things in this package, I think I'll go for it.

It is kinda black magic, and I think it's not something people would expect. You can import other things just fine though:

    import sys

    def wat(): print("wat")
    def bar(): print("bar")

    wat.wat = wat
    wat.bar = bar

    sys.modules[__name__] = wat

...

    >>> import wat
    >>> wat()
    wat
    >>> from wat import bar
    >>> bar()
    bar

Edit: I think a better way would be to instead add __call__ to the current module somehow though.

This looks super useful, but I was wondering if I'm the only one bothered by this recent trend of overloading completely unrelated operators (here the `/` operator) in the name of legibility.

I do agree overloading / is an odd choice in this instance, but it’s a shame “is” can’t be overloaded! Realistically wat(foo) would be fine though.

One might suggest adding:

  try:
      from wat import wat
  except ImportError:
      pass

to your $PYTHONSTARTUP file to avoid the cumbersome import.

You can even add the inline importer in base64 that is pretty neat.

But I eventually printed that output, and put it in a dir I point my PYTHONPATH to so that I will always have it available.

Let's see if it sticks.

Wow, this is the kind of tool that would have been a game changer when I was learning python; being able to see what is happening under the covers is part of the critical path for me to learn a language, and native python debugging is underwhelming, at best.

Instead I just installed pry and became a rabid ruby fanboy, but this might get me to give it another go.

the author is using the Python inspect module of the stdlib under the hood to provide the functionality, of course with a lot of value-add.

see inspection.py in the wat module.

it has this on line 2:

import inspect as std_inspect

https://docs.python.org/3/library/inspect.html

> If you want to quickly debug something, you can use this inspector without installing anything, in the same session.

> Load it on the fly by pasting this snippet to your Python interpreter

The idea of having a project's readme include a full copy of the project itself as base64'd compressed data is pretty ingenious!

especially for a project like this where you may not have had the foresight to preload it into the environment where you most need it

rich.inspect has been my go-to for this. Comes pre-installed in a vanilla Colab kernel too

Right! I usually use rich's print [1] for this kind of thing too, which I chose after testing a few other libraries that do pretty printing.

1: https://github.com/Textualize/rich?tab=readme-ov-file#rich-p...

I wonder if there's something like this for Lua. It doesn't have Python's built-in conveniences for introspection like `help()`.

There were various internal Quality of Life modules during my time at amzn, but nothing public. I suspect that given enough time working with Lua, each team will create an ad hoc, informally-specified, bug-ridden, slow implementation of half of python.

I found https://github.com/kikito/inspect.lua, haven't tried it yet.

Python's biggest weakness is the lack of a function that shows you how exactly a variable looks like. the var_dump (php, ha!) works like wonders. The best library ever created for Python ;). and now wat is also looking to be a close contender for best helping hand.

>Python's biggest weakness is the lack of a function that shows you how exactly a variable looks like.

Doesn't the repr() built-in function do what you want?

It can also be customized by defining the __repr__() method of a class, IIRC.

Commonly done.

Nop.

Yup. :)

Read about repr() here:

https://docs.python.org/3/library/functions.html

and dunder repr, i.e. __repr__() here:

https://docs.python.org/3/reference/datamodel.html#object.__...

This is incredible

Looks great! And it would make for a great addition to Jupyter lab. I actually can't believe there is still no decent variable explorer bundled in

Is there similar tool for typescript/javascript ?

Isn’t the browser inspector more than sufficient? You get the interactive object inspector in the console, live expressions, log points etc.

For node you it’s possible to hook it up with the browser inspector as well

Don't know why you're downvoted, but browser devtools have the best object inspection around. I miss them all the time when coding Python.

Hooking devtools with Node is possible but could be a lot nicer.

Given the natural built-in introspection capabilities of the language, I never understood why something like that wasn't also a built-in part Python.

I mean the number of times I've bumped on json not being able to dump the content of an object is just infuriating given how flexible Python is.

I'm definitely giving this a whirl.

>Given the natural built-in introspection capabilities of the language, I never understood why something like that wasn't also a built-in part Python.

Somthing like that does exist, and has, from Python 2.x; the inspect module, apart from help() and dir():

It is somewhat lower-level than wat, which builds upon it. But it is powerful.

https://news.ycombinator.com/item?id=41072878

Or did you mean the inspect module itself?

dir(), help() and now adding wat to my toolbox to help debug!

Now available from AUR: https://aur.archlinux.org/packages/python-wat-git

Is there something similar for Javascript?

Re: Insta-load

Please don't do this. Exec'ing arbitrary, obfuscated code you just copied from the internet directly in a running environment is such a bad idea it shouldn't even be suggested.

At the very lease please put huge warnings around this section to let people know that it is a horrendously bad idea and they do it at their own peril.

This is so bad it should be nsfw-blurred and you have to click 2 buttons to even see it.

I'm not a Python dev. Why would they do this? This is giving vibes of malware embedded into npm packages.

Convenient way to quickly add extra debugging capability without rerunning. It isn't much different from the many `curl example.com/install.sh |bash` you see around. It's up to the user to check things out before running.

Yeah, the piping to bash is a tried and true method for various installers. People make a fuss about it, but we don’t see people getting owned that way often. I think with bash installers though it’s pretty trivial to just visit the link and read through the 100 lines of bash. So anything installed this way should be as simple as visiting the link and reading a short amount of code imo

The same people seem to be more OK with installing random .debs and .rpms which can of course arbitrarily run any code with root.

Can't you just install it in your environment in another terminal and then import?

Sure, but it's a few extra steps. It's been proven(tm) that many people prefer a single, simple thing they can just copy, paste and run, so they can get back to their main concern.

And some may not want it actually installed for whatever reason. Such as when there's no proper separation between dev and prod deps. (I'm mostly just guessing at this point though...)

Not everywhere that python is run has access to pip. Sshing into some locked down remote machine and needing to debug some script is a use case that comes to mind.

Pip install allows arbitrary code execution.

How is it any different than installing the package via pip ? Not only most people won't check the source before running the code, but there is also no way to be sure that the code shipped by pip is the one you read on GitHub...

gp has a leg to stand on only if they regularly audit the contents of their site packages. Otherwise you're totally right.

the gzipped payload is static, you are free to inspect the payload before eval()

Yes, the big warning disclaimer in that part of the docs is definitely required here.

That idea of exec'ing arbitrary obfuscated (compressed) shell code that's easy to copy-paste into a python shell is very helpful, mind.

I've had to debug issues in production with silently hanging I/O threads and my only access is via `kubectl exec`.

This wat tool and that "insta-load" idea pairs nicely with pyrasite for that very useful !!!DO NOT DO THIS, YOUR CONTAINERS SHOULD BE LOCKED DOWN SO YOU CANNOT DO THIS!!! step of copying gdb and the pyrasite library into a running container when all you have is a python shell to play with.

(This almost feels like an exploit, running shellcode after getting RCE :))

https://pypi.org/project/pyrasite/

Maybe it's just me but I just use a proper debugger. Debugpy and VSCode work fantastically.

In a previous company I set things up so that when there's an uncaught exception it will automatically start a VSCode debug session and connect to it.

Here's the extension: https://github.com/Timmmm/autodebug/

Unfortunately the Python part of that is not open source but it was only a few lines of code - should be easy to recreate. That repo does contain a C library that does a similar thing.

You might just say "why not just run your program directly with the debugger?" and yeah that is better when you can do it, but I'm working with complicated silicon verification flows that are often several layers of Python and Make, followed by a simulator (e.g. Questa or VCS) that itself loads the library you want to debug. Very difficult to make debugging work through all those layers, but it's quite easy to write some code at the bottom of the stack that says to a debugger "I'm here, debug me!".

It important to remind everyone that you can already do something similar via the build-in function help(). For example, if I run help({1}) I get the documentation for a set, running help(set.add) gives the documentation for the add() method, etc. You can even preview objects that you haven't imported by using strings(ex. running help('numpy.ndarray') will correctly give you the documentation, provided numpy is installed in your current python environment). It's pretty helpful and doesn't require installing anything extra

and the dir() built-in Python function.

Also most advanced IDE's do this better and lot more to troubleshoot/debug.

You're completely right. `dir()` does the same in terms of functionality. In fact, my tool makes use of `dir` under the hood. I just wanted to make it more readable, and to combine `dir`, `type`, `repr`, `len`, `inspect`, etc. into one easily accessible place.

even just

  print(name.__doc__)

works for many kinds of names in Python.

And vars().

help says too much, usually you'd like a quick overview slightly formatted and well classified, which is what wat seems to afford here

I'm normally a big fan of python operator overload hacks, but why are we doing `wat / foo` instead of just `wat(foo)` ?

`wat(foo)` also works. The author says he implmented the `wat / foo` syntax to allow you to more quickly use wat by avoiding parentheses

Probably so you can run a big complicated formula in the REPL and easily prepend `wat /` to it to inspect what's going on.

Same reason jupyter notebooks allow you to prepend '?' or '??' to inspect a variable (though not expressions, in that respect this syntax is better)

I get the reasoning behind it, but the choice of the division operator is still a little weird.

I believe it should be equally possible to overload the bitwise OR operator (|), which, for people used to piping in Unix, is probably more intuitive.

Wrong direction: you‘d „pipe“ wat through the object. Although TiddlyWiki users might feel right at home.

It could override the reverse bitwise or magic method `__ror__`.

Right, that would be a wrong pipe direction. Fun fact: It's already possible to do "wat | foo" or "wat << foo" (if you're more familiar with C++ iostream), it has the same effect as "wat / foo"

Looks like a great tool. Will start playing with it whenever I have to dive into an existing project again.

One thing I'd like to note, though, is that most engineers (at least around and including myself) would be triggered by the "Insta-load" example of executing base64 encoded (and thus obfuscated) code.

I was just about to post that it freaked me out a little, but wasn't sure if I was going to be the only one.

The insta-load is fine. You're supposed to save the snippet, decode and uncompress it, look at it to see what it does and that it's completely self-contained uncontroversial Python code and copy-paste your version that you bookmarked. It's super short.

not hard to see it

with open('wat_source.py', 'w') as f: f.write(zlib.decompress(base64.b64decode(code.encode())).decode())

just so that it is clear for everyone, including newbies here, does the insta-load code snippet work like this:

the author has encoded the wat module's source code into base64. the snippet shown decodes that back into python code, and then executes it on the fly, thus having the same effect as importing the wat module in the normal python way?

Yes, that has the same effect as installing the package. So if you feel uncomfortable, you can either install the pip package (and of course review the installed code) or review the decoded string before executing it. It's not that obfuscated anyways, it's still quite readable.

got it, thanks.

This is interesting. I'm wondering what compelled the author to use the division magic method for arguments instead of the more intuitive and commonly used approach to passing parameters.

Interesting choice indeed. It looks like it supports the traditional approach as well

> wat object can quickly inspect things by using the division operator (to avoid typing parentheses). A short, no-parentheses syntax wat / object is equivalent to wat(object).

I think it makes a lot of sense in the context of a live interpreter. Wrapping and object in help() or dir() is an annoying set of movements vs. just pressing 5 keys with no modifiers (Home, w,a,t,/). It makes its feel much more like a magic command without actually needing it to be magic. The dot modifiers are also convenient compared to passing kwargs imo.

The README suggests for easier typing in interactive use.

It's the parentheses that drove me crazy. As people already noted, it's for faster typing, at the cost of the division magic as you noted. If it's more familiar to you, it works with `wat(object)` syntax as well.

I love it

Reply on news.ycombinator.com