From Python to Numpy
labri.fr
labri.fr
One thing I will highlight that the author just touched on briefly, is that numpy combined with numba is really a phenomenal combination for dealing with very computationally intensive problems.
The folks at Continuum Analytics have really done a fantastic job building numba (numba.pydata.org), which JIT compiles a subset of python functions using LLVM, and is designed to work seamlessly with numpy arrays. Numba makes it much easier to speed up performance bottlenecks and allows you to easily create numpy ufuncs which can take advantage of array broadcasting.
My team and I looked at Numba a year ago or so for optimisation of a fairly large calculation, and found that the speed-ups were impressive where they worked, but were not consistent or predictable.
We used Cython for large parts, and while there was boilerplate and incantations, the gains were achievable, incremental and certain. The annotation tools were also quite helpful for identifying bottlenecks where Cython code could be effective.
Incidentally, once we decided that Cython was our go-to tools, we often wrote simple looping code rather than vectorised code because it was simpler to transition to Cython, alá Julia.
Regarding whether it would be appropriate vis-a-vis cython really depends on your application
First, Cython is fantastic as well, and my endorsement of numba doesn't take anything away from it. Cython is much more fully featured and mature, in the sense that you can really develop your own data structures and control flows. Pretty much anything you could do in C, you could do in Cython. I've written Cython and it also plays very nicely with numpy.
In comparison, Numba is much more limited. You are basically limited to using numpy arrays and matrixes as your data structures, and you really need to understand exactly what is going to be used prior to the jit loop or you won't be able to use it in nopython mode (which is where you get the most benefit). It also doesn't handle strings really at all. One fairly recent thing Numba does is allow you to use a list of a single type within nopython mode. Under the hood it handles the malloc for you.
My endorsement of Numba really boils down to ease of integration with existing python codebase. For me the "killer feature" was the ability to simply comment out the @jit or @njit decorator and step through the code like I would step through normal python code, then just turn it back on again when I needed it. The other was that numba gained the ability early in our adoption to chain functions together, so while you can't generate a numpy array in the nopython mode, for instance, you can generate a numpy array in a @jit function (object mode) outside of the nopython mode, then call the looping function (nopython mode) from that jitted function, and numba handles that seamlessly and cuts out a lot of the overhead. For us, our speed of development of a custom algorithm has really been helped greatly by Numba.
The other thing I will mention is that when I first started, getting LLVM to work with numba was, initially, a nightmare on different OS's. That has completely gone away with improvements in conda package manager now.
All that said, you cannot go wrong with Cython, it just has a little more of a learning curve and was a little tricker to implement in our codebase.
>> once we decided that Cython was our go-to tools, we often wrote simple looping code rather than vectorised code because it was simpler to transition to Cython, alá Julia.
If you're used to doing this with cython, you might find it even easier to do this with numba. This is how I develop all the time with numba now. I find that it's incredibly beneficial to step through it as though it was just regular python initially during algorithm and test development, then once the algorithm is right and tests pass, turn on the jit when ready. You sort of get a sense for what numba will accept and still have performant no-python mode jitting after a while, and knowing those limitations actually tends to cause me to write more modular code to take advantage of the speed boosts.
It also has simple-to-use openmp-like multicore parallelization, limited class support, AOT compilation and CUDA & AMD HSAIL support.
Anyway, Mr. Rougier's Matplotlib was both informative, concise, and beautiful. Actually, I think my appreciation for matplotlib came from reading his guide: https://www.labri.fr/perso/nrougier/teaching/matplotlib/
For cmd-f: mirror
Sure, I guess he means search, but he could just say that.
Or should I start talking using references to my keyboard shortcuts? I mostly use emacs, very often with custom bindings, or tmux, so it will not make sense to a lot of people.
But hey, it's a free world.
I see that the author has responded to a couple comments here. Thank you for your great work! It's always great to have a nice reference material with concise examples. I think this will be helpful to everyone(beginners and advanced python users alike)!
Topics include: modules/libraries, documentation, distribution, virtual environments, unit testing, methods/decorators, functional programming, optimization, scaling, RDBMS, and more.
I'm no hot shit, but it definitely moved me from 'middling hobbyist' to 'semi-capable dev'.
Next I plan on doing https://github.com/CamDavidsonPilon/Probabilistic-Programmin... or http://www.composingprograms.com/
> For example, can you tell what the two functions below are doing? Probably you can tell for the first one, but unlikely for the second (or your name is Jaime Fernández del Río and you don't need to read this book).
Edit: it's sketchup - there's a .skp file in the data/ subdirectory of the github repo for the book.
I've converted the rst files to e-book using rst2html.py + calibre instead.
And it shows, the theme is beautiful. Also some of the best ASCII diagrams I've seen. Worth a look at the source, even if you don't care about Python.
That's actually a feature, as lines that are too long are harder to read (https://en.wikipedia.org/wiki/Line_length). It's the same as why books actually have empty margins.
To me, the font is too thin to be easy to read.
I agree with you about the font being too thin, but the line length annoyed me much more.