hogu

ipython uses (last I checked) node.js or pandoc(haskell) to convert markdown to html in nbconvert. You shouldn't need node or haskell if you're not using nbconvert

the server is written in tornado

I want to address several of the comments. I used to work for continuum, but I no longer do. I didn't work on the conda team, but as a part of the other teams at continuum, and at my new gig I'm a heavy conda user

1. linux package management is sufficient - this isn't the case. linux package management couples your library versions with your distro verison. Meaning if I want to do something like try out my code with the latest pandas, or run 2 jobs with different versions of numpy I'm out of luck

2. conda went ahead and forked everything without regard for the community - this is completely untrue. We sat down with guido for the very first pydata at google offices in 2012 to discuss many issues, packaging being one of them. Guido acknowledged that the python packaging ecosystem isn't sufficient for the scientific community, and we should go out and solve the problem ourselves - so we did. Honestly on this point, you should just read travis' words about the issue

http://technicaldiscovery.blogspot.sg/2013/12/why-i-promote-...

Scientific users need to be able to package non-python libs along with python libs (think zmq, or libhdf5, or R) we need a pkg mgmt solution that sits OUTSIDE of python for this reason. You can think of conda as a little in between place between virtualenv and docker

OP where are you at? I've consulted in a variety of different places in the financial industry, the opportunities range from terrible, boring and soul sucking, to extremely profitable and very interesting - the financial space is actually quite diverse

WOOT!

what is wrong with IDEs like pycharm/intellij?

few things that do not completely address your points, but may help

- python 3 (.5?) is adding the @ operator for matrix multiply. Of course this only helps you if your company uses python3

- There are 2 packages (seaborn, ggplot(clone)) which sit on top of matplotlib and provide a much nicer interface for statistical plots

what hoops are you referring to? My phd work was all matlab, and I vastly prefer python for readability and practicality

almost every finance shop in nyc that is using python is using python 2.7. And yes - for new code as well.

It's not. some people are solving this problem with https://github.com/conda/conda. There are many ways to use conda for deployment. My preferred approach is the following:

For builds: - build a conda environment for your project - export your conda environment using conda list -e, and then take all the conda packages for those and put them into a single tarball

For deployment - bootstrap a base python environment using miniconda http://conda.pydata.org/miniconda.html - install that tarball of packages with conda install <mytarball.tar>

It's not as simple as a jar, but it's reasonably close

I developed this library ( https://github.com/hhuuggoo/thedoctor ) to help deal with this problem. My belief is that type validation is only part of the problem - being guaranteed about properties of your inputs and outputs is also necessary (for example, make sure the input dataframes have datetime indices, and include the following columns, or ensure that this matrix is nonsingular). I have worked on some of the largest python projects at nyc investment banks, I think I've seen what I can safely call the scariest python project in the world

There is also another library out there which approaches the problem a bit differently

https://andreacensi.github.io/contracts/

I frequently work with large enterprise code bases. With the dynamic nature of python, it can be hard to understand what exactly you're supposed to pass in to functions, or what they return. The goal of this library is to make it easy to express those constraints, as well as catching problems as far up the stack trace as possible

Elite programmers absolutely exist, but not because elite software developers are smarter or more brilliant than normal ones. Instead I think that the best software developers do the following

1. Ability to rapidly prototype responsibly - make good choices about which short cuts can be taken, and making sure everyone understands the implications of those shortcuts. There is some curve of doing things right and doing things fast, and the good people I've seen can walk this line appropriately given the business demands.

2. Ability to not get bogged down in writers block or analysis paralysis, which I've seen take out weeks of productivity in less experienced programmers (everyone gets this some time, regardless of experience)

3. Domain knowledge of the problem at hand - so that you know what you build will actually be useful. Everyone can be told what should be done by sales or whatever, but combining knowledge into one head rather than 2 is vastly more productive

4. Enabling the above 3 things for the rest of your team.

If you're ok with using conda (python pkg mgmt tool) you can do conda install -c wakari pandoc, and it should just work

what group are you with? we're working on some relevant stuff

http://continuum.io/

I can be reached at hugo@continuum.io if you want to chat about it

I grew up on Matlab. Entire PhD thesis work was all matlab. I started using python after leaving academia because it's much easier to put python into production, and I've never looked back. There are only 3 benefits to matlab. 1. Simulink, if you do that kind of stuff (I don't). 2. Some numerical algorithms in matlab are more efficient, but then some of the python ones are more efficient. 3. matlab array syntax is a bit more concise.

Python destroys matlab in all other regards. Once you've tried it for a while you'll understand the value of a general purpose programming language with advanced numerical capabilities

ok, some of this is ridiculous

- sushi is really easy to screw up, just buy really crappy fish, and alot of people do that

- there's awesome chinese food and vietnamese food in san jose, much better than most of SF, second only to LA in the states.

If anyone needs more color, Travis, the CEO of continuum and the author of NumPy just wrote a great post explaining why conda exists, and why virtualenv and pip aren't sufficient for the scientific python community.

http://technicaldiscovery.blogspot.com/2013/12/why-i-promote...

in the short term, you should check out spyder

http://code.google.com/p/spyderlib/

we ship it with anaconda

No - that's not crazy at all.

In fact we are working on (and open sourcing) similar ideas

http://www.youtube.com/watch?v=b0-4xtFeaT8

Hey Hadley, you can get anaconda without giving us any contact information.

http://continuum.io/downloads

Great article!

I think gevent zeromq is no longer necessary

http://zeromq.github.io/pyzmq/api/zmq.green.html

One approach I've tried recently is getting a digital ocean box and giving the prospective candidate access to the box 24 hrs prior, just so they can install whatever tools they like if they so choose, and then giving the candidate a simple problem to build on the box for 30 minutes.

Then we mute the phones for 30 minutes, so that people can work in peace, but we're around if they want to ask us any questions.

And at the end of the 30 minutes we discuss how far they got, and what they would do differently, etc..

We've only done this twice, but I think it was fairly useful both times. Haven't done it enough to know whether or not this is a good approach.

The best feature of emacs is running shells in buffers. For interactive work, this is supremely useful because you can easily search your history, grab previous lines/chunks of code, and edit them into future commands. I do this all the time for data exploration with python.

I don't actually think of this as a workflowy clone, i think of it as an org mode clone, but I think more people know what I mean if I say workflowy

what are the pathological cases one runs into with gevent?

the blog has been updated with some pypy benchmarks added (just as a gist)

from the blog:

https://gist.github.com/ahmadia/5638980

Update:

At the request of several commenters, here is a test script and benchmarks that we ran on PyPy and Anaconda Python (with Numba). The results are not tuned (I am not a PyPy expert!) so we did not post them in the blog. We’d be happy to look deeper into this with the PyPy developers. While PyPy is not currently installed on Wakari, we are looking at a number of ways we can install and support the PyPy community.

sorry if this is obvious - Is this useable in the browser? I couldn't find mention of this anywhere, But everything that is inolved (typed arrays) seem like they should work client side.

thx.

awesome WM + running gnome 3 services, so I can get the notification icons for nm-applet, etc..

that's so garbage, I remember back in the early days, they were closing down accounts used in bitcoin exchanges. Getting money into bitcoin exchanges was a huge problem.

Besides, with bitcoin, why use paypal? Is paypal going go try to act as an exchange between btc and usd?

I've had pretty easy success moving my bottlenecks into Cython. With the development of Numba, https://github.com/numba/numba, this is going to become even easier(I work for the company that produces Numba, but I do not work on Numba). I think one thing the scientific community does right, is that we keep our data in NumPy arrays. This makes it really easy to write extensions in other langauges because NumPy arrays are very transparent and easy to access in other languages (they're just chunks of memory with dtypes, and a shape)