Python Packages and Environments with Conda | Hacker News Reader

Python Packages and Environments with Conda

continuum.io

40 pointsivoflipse16 comments

I'm not knowledgable enough on the whole package thing to comment on how good this solution is but it looks appealing to me.

I've suffered with pip doing almost everything I need but then struggling to install scipy etc. Any competitor to the space that can alleviate the pain is most welcome.

I know Armin had a rant about this last year [0]. Would love to hear thoughts by people that understand this far better than me.

[0] http://lucumr.pocoo.org/2012/6/22/hate-hate-hate-everywhere/

I, too, have been bitten by python packaging. The clarity that conda provides about what it is doing, and what goes where, is a big improvement over the current soup of easy-install and (to a lesser extent) pip.

That said, I'd still rather have a leiningen equivalent.

Is this related to Anaconda (the OS installer on RHEL, CentOS, and derivatives?). I couldn't tell if it is. If it is not please consider not calling it Anaconda.

It is even more confusing because Anaconda is concerned with installing OS packages, configuration and initialization.

I can see this dialog happening:

-- Have you updated the package in Anaconda?

-- Yes. It installs the numpy package.

-- Ok cool. Make sure it also configures your yum repos.

-- What yum repos?

-- You know rpms repos?

-- There are no rpms in Anaconda?

> -- Have you updated the package in Anaconda?

While I agree that anaconda may be an inappropriate name, since it has been the Redhat, Fedora, RHEL installer for, I don't know, maybe 13 years, this isn't really a concern. You don't "update a package in Anaconda". You don't even interact with it ever again once installation is completed, and most lay-users would never know what anaconda is unless they pay attention to the fine print during install.

Unless you create and customize kick-start script for VMs for example. Which is what I do often. Then, yes you do update a package in Anaconda by adding it to the packages section.

Why can't modules[0] be used for managing different versions?

[0]http://modules.sourceforge.net/

I always found managing Python modules using modules to be awkward. Using it to activate entire virtualenvs might work better. Though that's only worth it if you're already deploying a modules environment.

pip works great. The entire problem has been that easy_install and setuptools will not get out of the way. They would not fix their screwups for a long time. Now they have been broken long enough that it doesn't matter if they did fix up. THE problem with packaging in Python is too many package managers. Please stop!

I agree. None of these myriad package managers does anything that pip doesn't do. Virtualenv + pip still can't be beat.

Can't the same be said of yum or apt or whatever system facilities? I used Python for a couple years with many third party packages and RPMs worked fine. Why does each scripting language need its own package manager? It's madness. They all have the same fundamental needs, don't they?

I'd like to just use apt, it won't work for me. At least with Python, pip easily integrates virtualenv, which isn't true of yum or apt as far as I know. Plus, python devs can package once for pip and have it work on any system, without having to package separate deb, rpm, txz, etc. This strikes the balance of having too many package managers for devs to support and having too many package managers for me to use. I've made my peace with a single package manager per language plus apt for my OS.

How well does yum work on Windows and Mac?

Actually, this particular one does a lot that pip doesn't do...

It manages dependencies that aren't python packages.

Furthermore, it allows multiple, side-by-side installations of said dependencies in a user's home directory. And it's cross platform.

None of the current solutions can do that... (Even including full-blown package managers, as they're effectively limited to a single platform.)

I'm not saying that pip is going anywhere, but they did have very good reasons for making yet another python package manager.

I'll concede the point that pip can't handle non-python packages. But it can handle side by side versioning by way of virtualenv. But if I need a specific version for a project I just install that version into my virtualenv's /bin/ folder and that becomes the version used whenever I activate that virtualenv. It's super easy and doesn't add to the cluttered package manager ecosystem.

Yeah, I didn't mean to imply that pip didn't handle multiple versions (though my comment certainly reads that way). I intended that to be a comparison of "full-blown" package managers with conda. (As were the user-home-directory and cross-platform comments.)

At any rate, pip is really fantastic, and I use it constantly.

However, being able to handle non-python dependencies is huge in the scientific world, where lots of python packages have a _lot_ of non-python dependencies. Widely-used packages (e.g. numpy, matplotlib, scipy, etc) don't have very many, and are well supported, but a lot of the less widely-used packages can be a real pain to build.

I'm excited about it because conda has the potential to replace a couple of virtual machines I currently keep around for this purpose. I'll certainly admit it's not a majority use-case, but if it works as advertised it will be very useful for me, at any rate.

I wish this were true, I really do, but it's just not true. Maybe for python-only packages. But, for the myriad of python extensions necessary for Scientific Computing with Python, pip and virtualenv are not enough.

We think conda is a useful tool, and have freed it for use by everyone.

Perhaps something like pip + a better virtualenv + wheel would be close, but we needed something that works now.

Reply on news.ycombinator.com