Make "Python" run faster by shifting all the runtime heavy computation to C-implementations... this article is missing the whole point
Plus, the "C-implementations" he mentions are available as readily usable modules (numpy), or semi-transparent jit/aot compilers only needing a few annotations (Cython, numba), not actual C you have to write.
Besides, isn't the whole point: finish your project fast with the language you know using whatever it makes available to easily speed your code up?
As opposed to: "be a purist and not use wrapped libs written in another language".
Who cares for that? Even if it comes up, it's to avoid the hassle of having to deal with an additional language, setup etc -- which for numpy, Cython etc is almost none-existent (as you don't need to actually deal with C).
And of course, despite the purity of Julia's "single language", the hassle of moving to a totally different language, which few use, is not yet stable in syntax, compiler etc, and has fewer libs, should also be considered...
As such, I do think the article misses the point somewhat. Of course, if there's a numpy function that does what you want, you'd use it in real life. But what if there isn't? The nice thing about Julia is that the function can be written in Julia itself, and fast.
> It is important to note that these benchmark implementations are not written for absolute maximal performance (the fastest code to compute fib(20) is the constant literal 6765). Rather, all of the benchmarks are written to test the performance of specific algorithms implemented in each language. In particular, all languages use the same algorithm: the Fibonacci benchmarks are all recursive while the pi summation benchmarks are all iterative; the “algorithm” for random matrix multiplication is to call the most obvious built-in/standard random-number and matmul routines (or to directly call BLAS if the language does not provide a high-level matmul), except where a matmul/BLAS call is not possible (such as in JavaScript). The point of these benchmarks is to compare the performance of specific algorithms across language implementations, not to compare the fastest means of computing a result, which in most high-level languages relies on calling C code.
I have been in this exact situation, a numerical algorithm that was missing from Numpy but the rest of the project is in Python.
The solution is:
1. Write a Python function that operates on numpy arrays,
2. Add a few Cython type declarations to loop variables,
3. Mark the source file as "compile with Cython at runtime", which seamlessly turns the Python function into a C library.
The end result was a 1000x speedup compared to pure Python, very close to numpy built-in functions working on similarly sized arrays. And it needed only about 5 lines of setup code and type declarations for a few variables - all the code could still be Python and use all of Python even in the compiled files.
But then they go and write their own sort instead of using the language provided ones when offered. All these show is that julia is apparently faster than incredibly unidiomatic python written by someone who clearly doesn't write python. Okay. That's neat.
Python users shouldn't run standard loops in any scientific computing. Only valid examples are looping over a set of http parameters in a web app...
In which case, the computation will definitely not be the bottleneck.
The author understands your perspective but he's deliberately using a different one. The idea is that a data scientist user would realistically use NumPy/SciPy optimized C libraries instead of writing raw loops in "pure Python" to walk pure Python lists that model matrices. Therefore comparing pure Python code (interpreted by the canonical CPython interpreter) to Julia is the opposite of his goal.
The article's title is: "How To Make Python Run As Fast As Julia"
The author wanted to write about: "How To Make Python _Projects_ Run As Fast As Julia"
But many readers insist that the article should have been: "How To Make Pure Python Code Run As Fast As Julia"
(The 2nd type of article is also interesting, but the author didn't write it and didn't claim to.)
The article's comment permalink doesn't seem to jump to his exact comment so I'll copypaste the text here:
>There is indeed a disagreement about the purposes of the benchmarks. I see at least two purposes at stake here.
>1. A user point of view, which is to see how t best accomplish things in a given language. It is the result of various tradeoffs, including this: balance the time and effort to code something with the efficiency you get. That's the view of most Python users reacting to my post. We don't mind using Python libraries, even if they aren't written in 'pure' Python. Actually, the massive set of existing Python libraries is probably one key reason for its success.
>2. A language implementer point of view, which focuses on how elementary language operations perform. That's the purpose of Julia micro benchmarks I think.
>If people do not agree on the yardstick they use, then the discussion is not going to be fruitful. This disagreement explains most of the comments I saw until now.
>I am using the 'user point of view' in my post
In this case the way the author shows it isn't the best one: he modifies Python code to be more realistic - that's ok, but doesn't he do the same thing for Julia? Obviously, writing a recursive fibonacci functions isn't the best way to implement it. Obviously, using caching can improve performance. But why not to apply these changes to both implementations?
Yes, I agree he didn't rewrite the Julia fibonacci examples the same way as Python.
My comment was speaking more to the usage of "optimized C libraries" in his benchmarks as being appropriate for his particular goal. (As response to poster hojijoji's objection to C-implementations.)
Yes, when the author didn't change both Python AND Julia fibonacci examples in exactly the same 1-for-1 manner, it does detract from his overall message because it invites nitpicking. (The nitpicking is reasonable if you're hyperfocused on that fibonacci example.)
Based on your other responses in this thread, you seem to want him to write Python-vs-Julia benchmarks that's suitable for benchmarksgame[1]. You have a valid perspective but that's not the article he claimed to write.
[1] http://benchmarksgame.alioth.debian.org/u64q/python.html
Because he wasn't writing about Python in a vacuum. In his very first paragraph[1], one can see that the article was a response to Julia's benchmark.
Your question could be reversed for the authors of julialang.org website and they could've restricted themselves to say "here are some ways of writing functions in Julia" -- without bringing up Python at all.
But the Julia folks didn't do that because ... people like to write comparisons to other things!
[1] see 1st paragraph that begins and ends with: "Should we ditch Python and other languages in favor of Julia for technical computing? [...] did the Julia team wrote Python benchmarks the best way for Python?"