Let's Write an LLVM Specializer for Python
dev.stephendiehl.com
dev.stephendiehl.com
"Numba works by generating optimized machine code using the LLVM compiler infrastructure at import time, runtime, or statically (using the included pycc tool). Numba supports compilation of Python to run on either CPU or GPU hardware, and is designed to integrate with the Python scientific software stack."
Incidentally, I evaluated a few tools for going "one level lower" than numpy just last week for a project at work.
Pypy was out since our codebase already migrated to python 3.
Numba failed to install on Ubuntu 14.04 via pip even after some prodding. I don't want yet another python package manager on my system.
Cython is it's own language; not Python or C. I don't want yet another language in my codebase.
For our codebase I decided to go with numpy's ctypes integration; a good example is way at the bottom of:
http://scipy.github.io/old-wiki/pages/Cookbook/Ctypes
For our (already mixed) python/numpy/C++ project, the incremental cost was just a few lines of boilerplate per function to make up for both languages (and ultimately the linker) lacking a proper language-level nd-array type.
For our project there were no new dependencies, and as a bonus, after heavily testing and benchmarking your .so file from python, it's still callable from anything with a C FFI.
Apt/yum can't track different sets of dependencies for different projects for you, but conda does beautifully and easily, while keeping environments documented (in terms of what's installed) and isolated.
Really, miniconda is an under appreciated gem
Note that I am not a developer of numba, just a simple user, so maybe I was just lucky.
Use virtualenv. Having python specific packaging system makes sense with virtual environments. Tracking dependencies with requirements.txt and installing them with portable package manager makes build of a project portable. There are limitations to the current state of pip, but the idea behind it is sound.
I am a developer, I've used VirtualEnv for every web app I've worked on, but now I use conda for handling my environments/package management because our software was ported over to it.
The reasons we did that were:
a) Every one of our users knows how to use conda (we build a scientific visualization app)
b) it got rid of a few thousand lines of CMake
c) Installation time is now maybe five minutes instead of 1+ hours.
d) We don't lose a week trying to get our stuff installed on a supercomputer
Our users are all scientists. They don't care about "best practices", they just want their stuff to work, and to be able to do their job. The level of technical skill varies wildly, but now a tool exists and they actually use it to segregate versions of their packages into separate environments. This is a freaking miracle; trying to figure out what's broken on a person in india's computer when they did a crazy from-source install in an environment we barely support was just about impossible. Now we say "do conda list and send us the output; also, you can get a working environment by installing whatever previous version worked for you" is a huge change (yes, build systems and package managers have been fixing that problem for decades, but this was the first one compelling enough to actually get scientists on board).
conda has been fantastic for us, and our users love it. They don't give a crap about what the python community has deemed the "right" way to do something.
Last github commit 13 days ago == "abandoned"??
Still a lot of value in the parts that are finished, and while it is somewhat beside the point, it is much more finished than most similarly ambitious projects in the tutorial/learning domain. It even has a license that allows anyone that has the time, knowledge and will to extend it.