Pytype – A static type analyzer for Python code | Hacker News Reader

Pytype – A static type analyzer for Python code

github.com

63 pointsfanf229 comments

I guess someone posted this in response to this HN post:

https://news.ycombinator.com/item?id=19473631

People tend to do that on HN.

So, for those of us who don’t use Python type checking, which of the 4 checkers do we choose? Google, Facebook, Dropbox, or the new one: Pyright?

Not to be a total smartass, but if your Python codebase is not that big yet, then the correct answer is none of the above- rewrite in a compiled language if you find the lack of static checks disturbing.

I have been using "type checked" python for a year and let me tell you, it is not even 10% as good as having a real compiler. Python type checkers are not even close to 100% accuracy.

If you're going to use a python type checker, use the one that has the most dev resources behind it. Which probably means, whichever company has the most internal python code, which is probably Google. So pytype.

But bear in mind, these things are only sensible when you've written so much python code that it's too late to rewrite in a better language. If you're in that situation, even though these checkers are far from perfect, they are a massive improvement over plain python.

I only asked for an evaluation of the current Python checkers, not a philosophy lesson.

In theory there's no difference between theory and practice, but in practice there is.

So, in practice which is the best current Python type checker?

Have you tried PyRight? It claims to be much better than mypy.

What are the issues with mypy + python compared to say, Java?

As far as I can tell, Pyright is designed only for the VSCode plugin? Or at least is a typescript project that parses and analyzes python code, so if you're using that stack already it may be a good choice, but otherwise it's more stuff to install. Actually, I'll say people with python projects shouldn't "target" it because it's not a pure python stack, and it would add friction/overhead for people who don't already have npm installed. It's just for people coding python with the VSCode editor.

As I mentioned in a comment to that post, I used pyright for a few minutes and was impressed. You get red-squigglies under unexpected-type & no-such method errors, and I saw green squiggly under a "value may not be set" in a function like:

    def foo(x: int):
        if x > 4:
            y = 1
        return y

The immediate visual feedback is great. The dependency on typescript is not.

Searching VSCode, I see no plugins for mypy & PyType, and the pyre-vscode plugin requires pyre-check to be installed, and a .pyre_configuration file in the directory? All I know is I couldn't get it to do anything, but pyright worked out of the box with no effort.

I haven't used this PyType project, but it looks like it competes with mypy. I like that it, too, uses your setup.cfg file for configuration (I'm all about single-file configuration projects). Someone else will have to speak to the differences between them. I'm sure they have the same functionality, so the differences will likely be in the specifics like what the error messages look like and how easy they are to configure. If you contribute to various python projects, I supposed it wouldn't hurt to have them both installed.

As for which to choose, mypy is Guido's project (or at least the one he's associated with) and until another is clearly superior, you can be pretty certain it will track changes in the python language closely for a while. I use mypy & flake8 for "static checking".

My question is: are there any plugins that show error-indicator squigglies for pep8 violations?

Is pyright's dependency on node really that bad? Many projects need to have Node anyway, and/or use Docker or another tool to dictate which version of Python they're using.

As with everything, depends on the audience. If working on something web-related or has an obvious need for javascript, then yeah, node can be an assumed dependency (actually probably preferred because I'd imagine you get better integration with the rest of your project than with unittest/pytest). But if it's a pure-python library that has nothing to do with javascript/typescript I think it's too much to ask someone to be sure to install proper versions of node+npm+typescript+whatever-else so your `python setup.py test` runs correctly. The same would apply to a javascript project that had dependency on python3.6 for their type checker.

Many projects need node, but not all; I don't know any which require docker.

I want to be clear: with pyright, it's not bad! I think it's the right way to do the LSP plugin: no dependency on the current python version (am I using the python in the venv? or ~/.local/bin? or /usr/bin/?) and great integration with the editor.

If it's a large project that will benefit from "5x faster" typechecking than mypy, then yeah, it's probably worth the dependencies.

If you have a 5-file python package I found on pypi that I want to contribute to, but there's nonstandard hoops to jump through to get `python setup.py test` working properly, I would have preferred you made a different call in this regard.

I'm glad we have choices and it's not mono-culture.

Agreed; there are some projects, like pure-python libraries, for whom Node would be a dealbreaker.

Re: VSCode: Just select mypy as your Python linter and it will do all those red underline thingies to report errors in place.

Or set flake8 as your linter if you'd prefer to have PEP violations displayed instead.

Thanks! It's been in front of me this whole time... if it was a python, it would have constricted me.

Actually it looks like I can get flake8 & pyright simultaneously; that's actually the ideal situation I was looking for.

Pytype is the result of "hey, how hard would it be to run mypy on Google's code base?" Well, it turned out it was "take the idea and implement from scratch" hard (something that happens all the time here). I'm happy to have pytype have my back if I move on, but for my hobbyist coding I don't see much reason to switch from the mypy+pylint setup I have already wired up.

Nope. Our pytype project started before Mypy even existed. It was inspired first by https://developers.google.com/closure/compiler/'s success, and soon after inspired by TypeScript being released by Microsoft. MyPy came on the scene later in the same year we has started to work on early (unsuccessful) versions PyType. We didn't release pytype until ~2015 and didn't focus on making it usable in the OSS world until the last couple of years.

Pytype started with larger goals: It focused on static analysis and type inference; much more so than any of the other Python type checkers today do.

PyType, like MyPy, is also capable of analyzing Python 2.7 code because existing codebases have a ton of that and understanding types can help when porting it to 3. A couple years from now will anyone care? We hope not!

Performance is a problem for dynamic language type analyzers. Particularly so for Python where CPython is slow yet analyzers want to be self hosted in the language they're written to analyze. Very interesting, though not wholly surprising, to see Pyre and Pyright choose to implement in other faster languages. MyPy also has MyPyC internally which is doing a very Cython-esque translation of some of their performance hot spots into CPython API C code for a speedup.

Interesting times.

Oh, interesting. I've heard about mypy so much earlier that, given the similarity, I always assumed pytype was inspired by it.

Or MyPy?

mypy, although under the umbrella of the python foundation, is (or used to be, anyway) primarily developed by Dropbox

From the readme of Pytype, it's doing type inference instead of type checking. Although I don't know how accurate it is, type inference is much harder than type checking, especially on a language designed with dynamic typing in mind

It's pretty good. It's been designed to deal with the humongous legacy code base in Google.

I'm not an expert coder by any stretch of the imagination, but the linter built into Pycharm does a decent job of identifying the most egregious typing errors, IMHO.

I think a lot of it really comes down to how much domain knowledge you have with respect to the problem you're trying to solve. I tend to have to process a lot of data from unknown sources, with unknown and inconsistent encoding.

Most errors either show up immediately in Pycharm when I'm coding, within the first few moments of running the program, or two weeks later when it encounters a bizarre batch of records in the 15 millionth row of the database I'm processing or some strange network error that I neglected to foresee.

I personally don't see how compile time typechecking would make any difference to that latter set of errors (which are solely due to my lack of foresight on all the weird ways people find to handle data), and the first two sets of errors are usually resolved in the testing phase of initial program debugging.

I can't find the reasoning why is this better than mypy? Why not contribute to mypy instead of a new tool?

agree, this project even has a FAQ and I would imagine this should be the top question in it.

The docs say that it works without type hints. Looking at the code base, it is far too large to contribute to another project that appears to be quite opinionated.

mypy can also do type inference

not to the same extent.

Got any example?

mypy doesn't work with python 2

Yes it does: mypy can analyze Python 2 code - https://mypy.readthedocs.io/en/latest/python2.html

MyPy itself _runs_ under Python 3.

See also yesterday's conversation about Microsoft's static type analyzer for Python[0]

[0] https://news.ycombinator.com/item?id=19473631

> Windows is currently not supported.

Reply on news.ycombinator.com