Google has quietly released a static analysis framework for Python | Hacker News Reader

Google has quietly released a static analysis framework for Python

sourceforge.net

64 pointsrictic9 comments

it was an intern project from last summer, written from scratch by our intern Yin Wang

:-)

It's Steve Yegge again. Nice to see him not totally gone in the google black hole.

awesome! i really hope projects like this gain traction and are actually used by hackers. staunch supporters of statically-typed languages often criticize the lack of IDE/static analysis tool support for dynamic languages like python/js/ruby, citing that it's simply harder to create a good static analysis for dynamically-typed languages. while this is true in theory, in practice type, type inference and local dataflow analyses work pretty darn well, since people aren't just using types willy nilly in these languages --- there's actually a lot of implicit structure

Quite. This'll be an awesome addition to the python toolbox when a diagnostic tool is built around it.

Just in case anybody else is new to jython and wants to give Steve's HTML indexing demo a go:

  $ mkdir google-indexer && cd google-indexer
  $ wget http://bugs.jython.org/file756/google-indexer.zip
  $ unzip google-indexer.zip
  $ svn co https://jython.svn.sourceforge.net/svnroot/jython/trunk/jython
  $ cd jython
  $ patch -p0 < ../google-indexer-patch.diff
  $ tar zxvf ../google-indexer-newfiles.tar.gz
  $ ant jar

And then (following the suggestion in the top of the HtmlDemo class - see src/org/python/indexer/demos/HtmlDemo.java for how to do this on the whole stdlib)

  $ java -classpath ./dist/jython-dev.jar:./extlibs/antlr-runtime-3.1.3.jar:./extlibs/constantine.jar org.python.indexer.demos.HtmlDemo /usr/lib/python2.4/email

This will produce a set of files in html/. Use a suitable browser to view

  $ google-chrome html/Message.py.html

Some time ago I looked for something similar and found Pyflakes, which uses Python's compiler module to do some simple checks. This looks more promising. I have patched Pyflakes and I usually use it when refactoring code (check out http://amix.dk/blog/viewEntry/19361 for more info).

There's probably an interesting "summer job" cross-breeding this with Mark Dufour's ShedSkin Python-to-C++ compiler ( http://shed-skin.blogspot.com/ , http://code.google.com/p/shedskin/ )

By the way, I see ShedSkin 0.3 is just out this week. Mark has been persistent and seems to have interesting results.

"My team at Google also plans to continue work on [the analyzer] on a part-time basis, as it's an important part of a larger project we're doing in this space."

Interesting... I wonder what Unladen Swallow or App Engine could do with type inference and static analysis?

I think the larger project has to do with the language-neutral framework that he's advocated for (and is now apparently building). Google's now definitely got static analysis for Python, Javascript through Closure Compiler, and Java through a ton of open software (and probably a bunch of google-specific stuff too). Interesting to see where this goes.

don't projects like pydev (which has been completely OSS for a while now, and is even written in java, like the code Steve attached) already provide this?

Reply on news.ycombinator.com