Introduction to Parallel Python with IPCluster and Wakari | Hacker News Reader

Introduction to Parallel Python with IPCluster and Wakari

wakari.io

41 pointscorinna17 comments

Quick sincere question. I build HPC in the cloud tooling with AWS, MIT Starcluster, IPython, etc for a living. I'd like to give Wakari a try, but could the Terms compromise my ability to offer a product in the future that could be construed as a competitor?

Hmm, our goal is not to restrict what users can do. Can you email support@continuum.io with the specific clauses that are troubling you?

Thanks Peter, I'll do that. I'm a big fan, I was mainly thinking appearances on my side, like "hey you integrated Gate One shortly after signing up for Wakari." For reference this prompted my inquiry:

"Customer shall not ... access the Service, in order to build any competitive Service or product, or copy any ideas, features, or functions of the Service."

IPython is pretty unusual. When I was first introduced to it, I thought of it as mostly an upgrade for the default python shell. But as I find more and more features like IPy Notebooks and IPCluster, I'm realizing that it's a completely new sort of thing. It's not just better command history in your shell or whatever - it's a whole new approach to computing emphasizing reproducibility and distributed workflows.

I honestly thought the same thing until I was introduced to the scientific computing world last summer. I will have a blog post soon about how the differences of developing in a datacentric / ipython enabled python environment. The biggest difference is that the IPython notebook makes it reasonable to edit multi line functions at a repl.

Indeed, and communicating that clearly and effectively is something that we don't always succeed at. This is the reason why, on many occasions I had to resist fairly vociferous requests that we split up the parallel computing code from the shell, and also all the clients.

In the future that may make sense, once our internal protocols and APIs are so stable that we can do so without breaking functionality just as a way to help decouple development. But as a way to think about computational workflow, these pieces are all part of a coherent vision. It's just one that we don't always communicate clearly :)

How many people on news.yc are familiar with IPcluster?

Note: I work for Continuum Analytics.

Hey guys,

would you be OK if we use the image at the top of that notebook in the official IPython docs/tutorials/talks? We haven't made one that nice yet :)

If you have the svg sources, that would be awesome...

That should be fine. Thanks for building IPython, it's great.

Glad to be of use :) Do you have the original SVG sources for the image?

OmniGraffle images. I'll send you a zip and also the SVG.

I am not familiar with IPcluster. But I have used pyro and execnet. I think I will continue to use execnet, since it seems more lightweight than IPcluster. But the web interface provided by ipython notebook is really attractive..

I use it all the time!

Note, there is sometimes an internal server error when loading this page. If you reload it should work. There are some bugs on our end that we are working out

Scrolling is broken on the site for iOS.

Thanks for the report. Out of curiosity, are you accessing from an iPad or an iPhone, and are you using Safari or Chrome?

Broken on an iPhone running 6.1.4 on Safari and embedded webviews, for me. There also appear to be some resources which never finish loading.

Reply on news.ycombinator.com