Test for lists in Cython
github.com
github.com
It's better and easier to use Julia as your main top-level "glue" language and call Python/Rust/C from Julia. Julia is in many ways a better glue language than Python - better multithreading, easier calling into C/Rust, etc. Then, over time, to the extent it is practical, you can replace foreign code with Julia code - because Julia is fast enough that it actually makes sense to do so.
If you already have a large Python code base and can't switch the top-level language to Julia, I would just not use Julia for that project.
Why would you want to mirror the Julia repo instead of just downloading already-compiled binaries? Do you want to maintain your own fork with your own set of patches?
I went far down this path before infosec agreed to give github / highly respected cdn access.
I understand how can this be inconvenient for me and you while scripting. But this is absolutely no problem for julia's currently main target market who runs multi-GB simulations.
Also it is not like they will be spawning a julia instance in a hot loop (THAT would be horrible). You write some julia code, and import it using smth like pyjulia at the beginning of the file.
cpdef float iterate_list(a_list):
cdef double count = 0
cdef int i, j
for i in range(len(a_list)):
internal_list = a_list[i]
for j in range(len(internal_list)):
count += internal_list[j]
print(count)
return count
cpdef float iterate_list(list a_list):
cdef double count = 0
cdef double val = 0
cdef list ilist
for ilist in a_list:
for val in ilist:
count += val
print(count)
return count
In [9]: %timeit list_cy.iterate_list(a_list)
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
385 ms ± 6.15 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
In [10]: %timeit list_cyo.iterate_list(a_list)
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
1000000.0007792843
2.71 s ± 182 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
Where list_cy is the fixed code and list_cyo is the original code. Even then, iterating over a list of lists is _definitely not_ the optimal way you'd face a problem of this kind. Numpy arrays and memoryviews would be the correct tool to use.
Will Julia advocates every use honest benchmarks to make their language look good? I doubt it.
According to OP, he tried this but it resulted in slower execution for him. I'm not sure he really followed what that PR says. In the case of the submitter, it says on their machine it gave a 2x speedup (smaller than my ~7x, but still significant).
Like OP, I observed that static typing make_list yields little benefit. It's runtime is 3x the runtime of iterate_list. And that makes sense. I'm not sure why we'd expect Cython to speed up the allocation of large numbers of Python objects.
Another funny thing: making a pure python version that uses the sum() built-in results in code that runs 80% as fast as the cython version, but also gives lower error (e-8 versus e-4 error that gets accumulated when the numbers are summed up one by one).
Namely, realizing these claims requires learning the language, and actually taking advantage of it's strengths rather than just writing 'python in julia'.
It's very common for Python users to show up on the Discourse forum and complain that julia is slower than python and then show some code that's basically just Python code written in julia, including a huge proliferation of global variables, allocating huge amounts of temporary arrays, etc.
There's also a huge spectrum of 'benchmark quality' out there. E.g. the benchmarks this HN post features seem pretty shitty and are not measuring anything interesting or useful as far as I can tell.
Even if it doesn't say that, bashing people who use Julia for a repository made by a Python user is a new level of HN trolling.
There is multiprocessing module but you cannot share memory between processes. I guess the best options are either writing a C extension or using Numba. Writing C extensions require either distributing binary packages or C compiler to be present on the target computer which is not always ideal. So is using Numba the best solution currently? I tried it a bit in the past but the errors I got was a bit hard to interpret compared to regular Python errors.
I wish the multithreading module had support for native threads. Is there any PEPs trying to bring easy multi-core support for Python?
This can be an absolute sinkhole for performance, but if you have a large dataset you shouldn't be sharing lines that much one would hope.
Also, if you are running on Intel you might have some luck with Intel's profiling tools as, although I've never used them for python they are by far and away the best things in this area, they do support Python.
I don't know what details they give you in from Python code directly, but if you've used perf to profile code at a low level before, vTune is like a Tom & Jerry shotgun in comparison.
Never seen this before. Thanks!
The implementation may still be a bit buggy though, so use with caution! https://bugs.python.org/issue38119
Same thing in the R ecosystem, it's possible to use multiple cores with the parallel package but there are caveats too.
Also a big downside with JavaScript. Of course both Python and JS are high-level interpreted languages where high-performance use cases aren't the foremost priority.
Can't Node run multiple processes (or multiple workers, not sure of the proper terminology) on multiple cores? As discussed in this thread on StackOverflow: https://stackoverflow.com/questions/61893497/node-js-on-mult...
Of course, but you can also run multiple Python processes.
Modern JS engines are all primarily JIT-based, generally only interpreting code when it's faster than waiting on the JIT. JS JITs are quite good now, and in many cases will produce optimized C-equivalent compiled code from fairly naive JS.
Dask is great if you’re processing large amounts of data, and it recommends and supports threads for this reason.
Not sure there’s anything to prevent you from foot-shooting if you’re operating on the same array. I’d expect that this is only safe to do if, as you say, arrays are not overlapping. Let me know what you find.
The main selling point of the library is automatic differential and compilation to XLA, but I've been using it even when I don't need gradients, as it's really fast (due to compilation). I also really like the random number generator as it's very good for reproducibility.
I've played around with Julia in the past and really liked it, but in terms speed Jax has pretty much solved that problem for me
You may find this interesting https://github.com/scikit-hep/iminuit/blob/develop/tutorial/...
TBF I am mainly using it for mostly pure path functions.
Edit: also note that a big part of the umap library is written in numba.
It takes a bit of getting used to but the performance gains on impressive. Basically, my bottlenecks shift from compute to i/o.
I think you have to balance it against writing in C/C++. Mentally, it is basically the same work as writing in C (you manage memory/you write complicated for-loops) but you have good array support with numpy. The primary advantage for me that everything stays in the python runtime environment. You just run the code without any extra steps.
...
What is missing from the timing type 'toy' benchmarks is an understanding that there is typically more than one bottleneck in a real problem and it is easy to choose the wrong one to optimize and get little gains.
After starting C (30 years ago now), spending a long time in C#, then switching to Python a few years ago, I think the unappreciated advantage of python is that I have to abandon all pretense of caring about speed and just get stuff working. It basically solves the pre-mature optimization problem for me by being a fast interpreted language rather than a slow compiled language.
I feel the same way. With Python I just write the simplest algorithm that first comes to my mind, even though I know that it is not the most optimized way of doing things. But most of the time I am surprised that it works so fast that I realize I actually don't need to optimize it.
And being able to create and easily manipulate dictionaries and tuples also allows me to create efficient data structures very quickly.
With Jax you can write native for loops that can also be jitted (I imagine you can also do this in numba?); this can then be really fast. Though in that case you would have to write the optimisation algorithm yourself which is not always practical!
Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation. For example you can build a massive gram matrix really quickly using vmap.
Another point: Jax can also run on GPU (like numba :) ) without having to rewrite anything.
Yes, you can just pass the function and it runs the optimization, in severe cases, you can start by doing a grid `.scan` or `.simplex` (Nelder-Mead simplex method), then `migrad` to minimize, and `.hesse` for 1 sigma bound.
You can also provide a gradient function that, well computes the gradient instead of computing it numerically.
> I imagine you can also do this in numba
Yes! It compiles to native code.
> Another big speedup in Jax is due to vmap/pmap, which allow to vectorise/parallelise computation.
It's possible to compile some vmapped functions with numba too, if the link is any indication, you may see even greater speedup than just jax.grad
However, I do concur that jax's vmap is absolutely fantastic and I found it very useful on many occasions.
I understand the sentiment, but in this case I wonder if it is less a python issue and more an issue with numpy. I don't see any reason why numpy couldn't execute the last line of the following multi-threaded:
>>> import numpy as np
>>> a = np.arange(1000000000)
>>> b = 2 \* a
edit: Actually it seems numpy is multi-threaded under certain circumstances:
https://stackoverflow.com/questions/16617973/why-isnt-numpy-...
Distribute your data and run everything as dask.delayed and then compute only at the end.
Also check out legate.numpy from Nvidia which promises to be a drop in numpy replacement that will use all your CPU cores without any tweaks on your part.
I tried to run the benchmark on my own computer, but the setup documentation was not enough for me to get the julia integration running. I haven't used julia before, so it might just be something very simple.
Similar I haven't used poetry much before, and the given documentation failed to install the necessary setuptools-rust for me. I could fix it on my own, but doesn't make me feel certain about the outcome of the benchmark.
The rust benchmark did not reproduce for me: "Rust (Pyo3) parallel after" has a 1.56 speedup for me, but a 2.6x slowdown for the author. Also I don't understand what the difference between "after" and "before" is, the code just calls the same code twice. Might be a JIT/Cache thing, but it's unclear to me. One sentence what before/after refers to would be very helpful.
Generally all measurements are only done once. Measuring at least thrice gives one at least a chance to detect an outlier and gives possibility for statistics, e.g. is a difference betwee the different cython annotations even meanginful?
The "C Cython (pure-python mode)" is reported faster then "C Cython (.pyx)". The Cython project itself says that using pyx files should be faster, so something strange is going on.
"Cython is fast, but none of these methods are able to release the GIL. " (A) this is not true (B) this seems to be mostly over single threaded performance, so why is that meaningful?
"Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library." The rust code already contains Pyo3, so an "added library" is not necessary as far as I understand.
I'd guess the performance stems more from conversions between different types then anything else. Maybe Julia (and the python-julia bridge) is particular smart about it and thus it's super easy to use, while pyo3 (and cython) needs some more work to interface with python. Even if that is true, I couldn't say it from the presented data.
With these caveats resolved I'd be interested in the benchmark, but without it I can't really say anything from it.
1) Calling into an actual static language like C or Rust is the best option. You get maximal performance and all the benefits of the static language. The downside is that you need to learn another language, and manage both languages in your project, including setup and compilation of the static language etc.
2) Cython is easiest for small-scale projects, since it integrates very well with Python, and you can learn it incrementally. But I found it annoying to work with - it felt like half a language that fell between Python and a proper static language. I ended up using Cython in the end, but I wasn't happy with it.
3) Numba looks interesting and promising. At least 2 years ago, it was too brittle and had too many situations where it didn't work or didn't give noticable speedups. I'm sure they improved it since then. I would definitely take a look.
You can always just learn Julia of course and have this entire problem of "my high-level language is too slow" completely disappear ;)
Fixing those minor issues cuts the runtime in half (see #3). Going for actual high-performance Cython (on my 12-core workstation) cuts the runtime by 20x (see #2).
https://github.com/00sapo/cython_list_test/pull/2 https://github.com/00sapo/cython_list_test/pull/3
On the other hand: having used pyo3 to integrate rust with python in the past, the biggest pain is simply to reconcile the dynamism of python with AOT compiled rust code. There's a lot of noise at the interface from the large amount of type checking to unpack the specific types of numpy arrays coming across. Do you want to have your rust code work with all sorts of integer bit lengths? That'll be a code path for each. If you have a number of input types, have fun coercing them all.
This can be alleviated with macros to a degree, but that just hides the problem really.
Julia is JIT compiled, which means that there are no problems with types being determined only at run time. In fact, this one fact makes integration at the code level much nicer with Julia.
So all in all it's a tradeoff really. I also found the tooling for Rust to be much more robust and stable (1 year ago admittedly).
After using Ruby for years, returning to Matlab style code in Julia felt somewhat awkward. Instead of my_array.length you have length(my_array). In my personal preference the method call is just a nicer way of doing the same thing. The single-instruction-multiple-data or dot notation sometimes worked and sometimes didn't, so you had to resort to loops anyway.
% Julia
a=[1 2 3 4]
a.^2
log.(a)
# Ruby
a=[1,2,3,4]
a.map{|element| element.pow(2)}
a.map{|element| Math.log(element)}
Other languages of course take some of these things even further. Maybe some day I will find an ergonomics and nicety first successor to both.
Plotting with two Y axes or generating histograms in Julia was also way harder than I remember it being in Matlab. Also manually having to load the file before every run to see the changes in action added a lot of overhead to the workflow.
The workflow I was used to in Matlab involved very frequent making changes to code, running of the code that usually made some plot. Which is one command in Matlab, and plotting was fast, in 2001 already, on Windows NT 4. In Julia, you have to first load the modified file, then run it. Plotting takes a long time. One just can't get nearly as productive with it in 2021, compared to Matlab of 2001 vintage.
What language would I pick if I had to do some quick analysis from some tables downloaded from the internet? Probably Julia still. If I had free access to Matlab, I would probably use it though.
Well...they're both method calls, aren't they? (So they're both the nicer way?)
>>> ', '.join(['1', '2', '3'])
'1, 2, 3'
>>> '1, 2, 3'.split(', ')
['1', '2', '3']I read the first as the concat symbol applied to an iterable leads to string concated by the concat symbol.
I read the second as an iterable broken by split symbol leads to an iterable of the chunks.
In the first case, I want to do something to a list: join it into a string. So, clearly, I need a list method? But no, I need to engage in some form of indirection; for some reason, I need to reach for a string method. Even if there is no string that I want to use as a delimiter. In that case, I need to use a string method on an empty string.
OK, I’ll play along. Now I want to take the string and split it into an array. Now that I’ve been educated, I know better than to try the sensible thing. Pre-enlightenment, I would have reached for a method applied to the thing that I wanted to transform. But now I know I should think backwards, and use a method applied to the delimiter. OOPs.
julia> join([1 2 3], ", ")
"1, 2, 3"
julia> split("1, 2, 3", ", ")
3-element Vector{SubString{String}}:
"1"
"2"
"3"
You'll have to decide if you find that convincing. I understand your point and kind of wish I hadn't read this thread because I'm more torn than before.
https://arstechnica.com/science/2020/10/the-unreasonable-eff...
Not in CLOS, from which Julia's mechanisms are derived (which leads to the question why not CLOS but Python of all things should be used as a source of "normal sense" of anything Julia-related). In CLOS, classes bind together data, generic functions name abstract operations, and methods represent specific code that deals with implementing a generic function for a particular combination of type arguments.
> which is what leads to all the problems
I don't think anyone is disputing that here. There's a reason why CLOS didn't do any of that.
> In Julia functions are generic, and specialized to methods acting on arbitrary combinations of types; they are freed from the data.
...yes, just like in CLOS (unsurprisingly, given Julia's heritage), and those types are effectively CLOS classes.
Although multiple dispatch in CL (and Perl) predates Julia, I was not aware that Julia’s design derived from it. Do you have a reference that traces this?
Interestingly enough, in Julia's documentation, the section "Noteworthy Differences from other Languages" (https://docs.julialang.org/en/v1/manual/noteworthy-differenc...) compares Julia to only several relevant languages, which are: Matlab, R, Python, C/C++, and...Common Lisp, of all things. I very strongly doubt that this is a coincidence.
This could have easily been avoided if `join` would have been a method of the list object, not the string object. But we're stuck with this now.
Maybe it would be better as a standalone function. But then you'd either have to import it or it would pollute the global namespace.
Other OOP models offer different solutions.
`map(el->el^2, a)`
and inspired by Ruby
``` map(a) do element log(element) end ```
The latter being syntax sugar for the former.
Using the latest version (1.6 - although 1.6.1 just came out) the time to first plot is just a few seconds. After that, plotting in the REPL is instantaneous.
I probably don’t understand what you’re getting at when you speak of making frequent changes to code. REPL-based development in Julia is excellent, and there are Pluto notebooks as well.
I would run the code in repl (just using up-arrow and enter), get some plot, modify the code in the editor, save it, and rerun it repl.
Repl is used for connecting inputs to the program, not for editing. Often times you want to develop some small program that is part of a bigger program. So you run the big program up to the point it calls the small program. Or you run with some mock data. Then you have all your inputs for the small program, so you can start iterating it. Run - observe - edit - run - observe - edit.
In Matlab this worked because it detected if the file had changed, and reloaded it. You didn't specifically have to call load("my_function.m") before every call to it. This means you can just lift code from the big program and iterate on it in repl.
https://docs.julialang.org/en/v1/manual/workflow-tips/#Revis...
https://github.com/urschrei/simplification
https://github.com/urschrei/lonlat_bng
https://github.com/urschrei/pypolyline
Each of those repos has links to the corresponding Rust “shim” libraries that provide FFIs for dealing with the incoming data, constructing Rust data structures from it, and then transforming it back on the way out.
As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
https://play.rust-lang.org/?version=stable&mode=debug&editio...
Because the data structure LotsOfData doesn't implement the Clone trait, attempting to create a copy will fail at compile-time.
https://run.dlang.io/is/jOXfJz is a D equivalent
When I was interfacing D code with a part of Unreal Engine that I think is garbage collected, I actually just took the L and copied everything into buffers on their malloc when handing stuff off to the engine. It wasn't particularly hot code so the memcpys were worth the peace of mind I found, ugly as it was.
> As a more general comment, using a GC language as the FFI target from a GC language is begging for difficult-if-not-impossible-to-debug crashes down the line.
<s>As for the FUD about interfacing JIT code to C/C++, this is a problem Julia was designed from the outset to tackle. Incidentally, Julia built on the excellent experience LuaJIT has had. A challenge: can you name a particular extension that would interface better from Rust than from Julia?</s>
Oops, I misunderstood the criticism you were hinting at but failing to justify, sorry. OK, Julia already does interface to Python and this interface sees widespread use. If your suspicion is right, then where are the horror stories from people who were bitten when deploying code that built on the interface?
Nobody claimed there was a problem?
> As for the FUD about interfacing JIT code to C/C++
I don't know what you're talking about. Sorry!
Not true!
What you do is that you keep a registry for objects passed from the host vm to the foreign vm in which you register objects thus transferred. And you use a similar mechanism for objects passed from the foreign vm to the host vm. In CPython, you simply increment the refcount to prevent Python from collecting them prematurely.
This is how Java does it (via JNI) and how many gc:ed runtimes interact with other gc:ed runtimes. This is how you do it in Rust too since Rust can't tell how long an object passed to a foreign vm is supposed to last.
It's not that easy. You'd need to register the object as a GC root as long as it's being managed by the foreign GC, and similarly ensure that objects in the foreign VM are properly "de-registered" when the local GC finalizes them. It's far from trivial, particularly when compared with other memory-management strategies.
It's funny the hostility to JIT that some fans of compiled languages have.
Same no copy, slightly more user friendly approach.
Further criticism of the actual approach - even if we didn't do zero copy, there's no preallocation for the vector despite the size being known upfront, and nested vectors are very slow by default.
So you could speed up the entire thing by passing it to ndarray, and then running a single call to sum over the 2D array you'd find at the other end. (https://docs.rs/ndarray/0.15.1/ndarray/struct.ArrayBase.html...)
Safe interop between two GC'd languages (Haskell and Java) was one motivation of Haskell's new -XLinearTypes extension
https://www.tweag.io/blog/2020-02-06-safe-inline-java/
Generally, the new linear types are exciting but are nascent at the moment. But one extension to them (linear constraint [1]) seems to allow embed the equivalent of Rust's ownership in Hsakell using more primitive features in the type system (to my understanding..)
I honestly think people are severely underestimating what a massive impact this is currently having in increasing productivity of python devs who know a little C++.
I don't know if this particular benchmark transpiles correctly or not, but it should be possible to achieve much speedup if you annotate your python code properly.
https://github.com/adsharma/py2many
Looking for help with open issues.
produces
Although as mentioned in the benchmark, with py2o, rust will be a superb contender, especially considering:
- the toolchain is the easiest to setup
- you can embed asm if you really need this extra juice
> Rust is not that fast because it needs to copy data;
I'm surprised.
Don't know much about Rust, but isn't it hailed as being competitive with C/C++ ?
The Julia implementation replaces the input list with a Julia object, which is of course trivial for Julia to iterate over.
The Rust implementation is asked to convert the list-of-list-of-floats back to Python types, and then back into Rust types. Of course converting 100M floats back and forth is going to take a non-trivial amount of time.
The equivalent in Rust would be to wrap the return value in a PyObject that contains the Rust `Vec<Vec<f64>>`, and the Rust code could then just iterate that directly. It would even be made safe without the GIL as you'd implement your own mutex on the Rust side if you wanted it to be mutable.
If you do this then it runs in about 1.2s on my laptop.
In general, rust is at least as fast as C or C++, but this seems to be a special case.
> Rust is not that fast beacuse it needs to copy data; using Pyo3 objects would probably lead to similar results as cython, but with an added library.
"It needs to copy data", because it's converting Python objects into Rust objects and back again. As the full quote states, it could be written in a different way, although that would make the code look really weird.
In this benchmark there is also no normal C++ involved. There is only Cython using some C++ ints, but operating on Python lists. And that's not handled by writing C++ code that operates on Python structures, but writing Python code in Cython in a "C++ mode".
Apples to oranges.
Not it wouldn't?
Using Pyo3 is arguably the most idiomatic way of interfacing Rust with python. The "added library" cost mentioned as an excuse not to use it is literally the 1 second it takes me to write `cargo add pyo3` .
They didn't do this because it would make their claim that "Julia is the better language for extending Python" kind of moot.
They started from the conclusion that they wanted to have, and worked their way backwards towards implementation and data that would support it.
I'll defer to your knowledge on that. I generally don't work with Python. I've only looked into interfacing C# with Rust and it looked crazy, so that's where I'm coming from.
But yes, it's pretty clear that the results are more a consequence of what's in the source files than the languages used (if we can even say that e.g. C++ was used at all).
AFAIK, there is no library that achieves a similar idiomatic interoperability between Rust and C#, like PyO3 for Python.
C usually comes out a bit worse because it gives even less information to the optimizer.
And thanks to the GNU/LLVM monoculture most languages should perform roughly the same if they have them as a target e.g. I have found that D makes its it ridiculously easy to write highly specialized code that is both readable and visible to the optimizer (I will be blogging about that part later), but I'm sure I could force the same asm out from C++ or Rust etc.