PyPi interactive dependency graph

It would be nice to be able to search for a package. Also, the graph currently has a node per "dependency", which may be with a pinned version. Maybe it might be even better if it just showed package dependencies? Looking through the structure you can see "interesting" packages, but because a package counts as different if a version is specified it makes it hard to browse through the interesting connections.

I was curious about what the most common dependencies were so I hacked the IPython notebook a bit:

  import json, pip, re
  from base64 import b64decode
  from itertools import chain
  from collections import Counter

  def get_name(x):
      "This captures most of the base names for dependencies"
      if x.startswith('#'):
          return
      try:
          return pip.req.InstallRequirement.from_line(re.sub('["\']', '', x)).name.lower()
      except ValueError:
          return
  
  C = Counter()
  
  data = []
  with open('pypi-deps.csv', 'r') as file:
      for line in file:
          name, version, deps = line.split('\t')
          deps = filter(None, map(get_name, json.loads(b64decode(deps))))
          data+= [(name, version, deps)]
          for x in deps: C[x] += 1
  
  print '\n'.join('{0}\t{1}'.format(*x) for x in C.most_common(20))

The results:

  setuptools	1949
  django	546
  zope.interface	244
  requests	241
  lxml	204
  zope.component	203
  simplejson	202
  flask	158
  zc.buildout	127
  argparse	110
  zope.schema	109
  python-dateutil	108
  jinja2	105
  distribute	104
  pyyaml	103
  fanstatic	95
  beautifulsoup	93
  zope.publisher	91
  sqlalchemy	91
  numpy	89

It would be interesting if one could trace the dependencies recursively - but knowing how PyPI works, that would be incredibly time- and bandwidth-consuming.

Still, if one had access to a mirror, it could be done...

My favorite is:

  [x.strip() for x in open("DEPENDENCES.txt").read().split("\n") if x.strip()]

setup tools and django win