Punchline: rewrote the code to look almost identical to C++, hand-held the compiler by adding @-marks to disable safety checks, forced SIMD codegen and fastmath on.
End result: code that is uglier and still much slower than C++. Kind of a shame.
End result: code that is uglier and still much slower than C++. Kind of a shame.
I wrote a blog post at the time with exactly that punchline (not explicitly stated, but just look at the code!): https://spmd.org/posts/multithreadedallocations/ The example was similar to a real production-critical hot path from work.
Maybe things changed since I left Julia, but that was December 2023, for years after this blog post.
This is the last step before I move to code generation and then generating a ton of test cases/debugging.
My goal is some form of release by the end of the year.
I've thought a lot more about the engineering than any sort of marketing or businesses plan, so I just want to defer those.
As a quick anecdote, in our take-home interview exercise, we usually receive answers in C++ or Julia, and the two fastest answers have been in Julia.
Of course it also depends on what additional libaries you are using, especially when it comes to parallel/GPU programming in C++, but easy to believe that Julia out of the box makes it easy to write high performance parallel software.
This only ends up being true (for any language, but it's too often cited for C++) in a pretty useless Turing Tarpit sort of sense.
So it's not "no reason" it's just sometimes impractical to solve some problems as well in C++ as in a language that was better suited.
Now people do do impractical things sometimes. It's not very practical to swim across the English channel, but people do it. It's not very practical to climb Mt Everest, but loads of people do that for some reason. Going to the moon wasn't practical but the Americans decided to do it anyway. But the reason even the Americans stopped going for a long time is that actually "that was too hard and I don't want to" is in fact a reason.
Yeah, I actually totally forgot to check the date...
So I would say that the culprit for interoperability is C and its descendants, not Fortran or Julia. The designers of C and of the languages that have imitated C have not given any thought about which order for multi-dimensional arrays is better, so the users of such languages do not have any right to blame for interoperability other languages that have done the right thing. Even if the Fortran order had not been better, it had already been used for 20 years before C, so there was no reason to choose a different order.
C has chosen to store arrays in the order in which they are typically read by humans when written on paper, but this is a choice like the choice between big-endian and little-endian, where big-endian was how Europeans wrote numbers, but little-endian is more efficient on computers.
An example of why column-major order is preferable, is the matrix-vector product, i.e. the evaluation of a function that maps linear spaces.
The matrix-vector product should not be done as it is typically taught in schools, by scalar products of rows of the matrix with the vector, because this is less efficient, by making more memory accesses. The right way to compute a matrix-vector product is by doing AXPY operations between columns of the matrix and the vector operand (segments of the output of the AXPY operations are held in registers until all partial AXPY operations are accumulated, avoiding memory accesses). In this case, you need to read columns of the input matrix for each AXPY operation, which is much more efficient when the elements of a column are stored compactly in memory, avoiding the need of strided accesses.
The same thing happens for matrix-matrix products, which must not be done in the naive way taught in schools, by scalar products of rows of the first matrix with columns of the second matrix, but it must be done by tensor products of columns of the first matrix with rows of the second matrix.
This is a plausible assumption to make but unfortunately it is not true at large. Especially when the traditional sizes are exceeded say n >= 2000 certain operations such as LU can be improved in terms of performance with C-major arrays. However the correct statement is you lose at some place you win at other. There are certainly linalg operations that F-major can give you more performance. However it is also true for C-major layout.
In your example matrix vector product or any BLAS2 or BLAS3 level operations you can also swap out the for loop order to convert things around (row*col buffer multiplication vs sum of weighted column sum interpretation). In particular matrix norm operations are the only exceptions (abs column sum, row abs sum etc.) that certain norms prefer certain orders. In fact if you go into the Goto method deep enough you'll see that internal order is a bit like Morton ordering to fit things into L1 Cache.
The reason why column-major is preferred is historical and requires more surgery to get it running with C-major ordering. Trust me I tried but it's too much work to gain not so much. Maybe someday when I retire I can attempt it. Hence I kept it column major in my retranslation of LAPACK https://github.com/ilayn/semicolon-lapack
Instead I implemented a "high"-performance AVX2 matrix transpose operation so that swapping the memory layout is trivial compared to the linalg cost.
Oh such a shame indeed! They didn’t even manage to produce better looking code at least?? Julia was looking great in 2019 but it was very buggy still so I stopped looking. Had hopes that by now it would be a good choice over C++ and Rust with similar performance.
I have always seen it as a potential alternative to Java, and definitely better than Python.
My experience working in it professionally was that it was... fine. But the GC in it was not good under load and not competitive with Java's.
The key to performance with the GC in Julia is not allocating, but it has gotten substantially better since 2019.
But interfaces are informal. Not using a monorepo say makes it harder to be sure if your broke downstream or not (via downstream’s unit tests).
But freedom from Rust’s orphan rule etc means you can decompose large code into fragments easily, while getting almost Zig-style specialisation yet the ease of use of python (for consumers). I would say this takes a fair bit of skill to wield safely/in a maintainable fashion though, and many packages (including my own) are not extremely mature.
I was never an expert in the language, but worked along people who were and they generally made nice code.
But there were a few places where I saw intensely confusing patterns from overloading with multimethods. Code that became hard to follow, and had poor encapsulation.
Also, I'm of course using nefarious in jest here in both cases. While we don't directly try to monetize our open source work, I respect that sometimes people need to do that. As long as people are transparent about it, I don't have a problem. Doing the thing we're doing seems to work, but it's a lot harder, because you have to build a successful pice of software and a (or multiple) successful something elses that has a critical dependency on it. It's like hitting the lottery twice.
Also, contributing in open source is a choice, not a mandate. I greatly benefit from Julia and its ecosystem so I chose to contribute back some of my work, no one forced me. I chose the MIT license because I want other people to be able to make money with it, just like I make money with other peoples MIT licensed stuff.
It's interesting. I like the more opaque approach rust takes. Rust has its own issues but it seems less corporately motivated. Maybe that's why it has more corporations using it? You aren't going to end up with the core maintainers to the language rug pulling packages or language features to slow down competition who are also using the tool. I say competition because it looks like they are making money through consultancies and very broad applications of the niche language.
Weird stuff to have to think about. I just want to write code
this is not true; the other comment is wrong. there is no central body at all that "decides" what features are prioritized. features are simply worked on by whomever has the capacity, ability, and desire to do so.
many engineers at JuliaHub have all three of the capacity, ability, and desire to work on certain features because JuliaHub, in its capacity as a private business, pays them to do so. but with respect to Julia the programming language these are "just" third party contributions like any other.
From a quick Google search it looked kind of like a bunch of MIT staff/professors(?) are getting students to churn out code for a variety of business interests. Just doesn't seem right in the surface and does make me wonder about what other things happen knowing what I know about human behavior.
I am personally not interested that's for sure. Thanks for sharing your experiences though.
I don’t if these are contradictory exactly but it seems to come from a very cluttered space.
It’s nothing like Google-the-ad-company influencing Chrome. The company consumes Julia for products to sell, rather. Maybe this affects the ordering of features landing, but… meh.