StefanKarpinski

The password is still revealed to the server. There are password verification protocols where the password is not revealed to the server either, which is much more secure as it means that you’re not at the mercy of whether the sever operator follows good security practices about not saving your password in plain text somewhere.

Good paper—it was the primary source we looked at when implementing these operations in Julia. Specifically, Julia implements all versions and overloads `div` and `rem` to take a rounding mode argument. It also gives special names to a few of them and they come in pairs: `div` & `rem`, `fld` & `mod` (`fld` stands for “floored divide” as opposed to `div` which rounds towards zero). The `%` operator does remainder like C, which was a tough call, but we figured being compatible with C was worthwhile for people porting code. It’s also less useful in Julia due to default 1-based indexing of arrays; for that there is a `mod1` function that returns a value between 1 and n.

That's not how Julia's JIT works—it's "Just Ahead of Time" which is a term we use to distinguish it from what other modern JITs tend to do, which is generate code after some interpreted execution based on observed behavior. What Julia does is much more akin to static compilation but done the first time you need the code, based on strictly in advance data flow type analysis of the code. One of the interesting things about the language that this paper touches on is that Julia and its libraries are designed so that you can analyze code well enough in advance to perfectly predict types much of the time, even though that is not a static requirement of the language.

That means it does depend on the value.

Ah, ok. I hadn't gotten to that part yet. Would it make sense to change the general definition to "expressions" since that would be the correct definition for Julia code as well as the Jules IR format?

This is one of the reasons that Julia doesn't do automatic big num conversion, instead using native machine two's complement modular arithmetic for integers. It would be a good optimization to use an immediate representation for small bignums, however. See this FAQ entry https://docs.julialang.org/en/v1/manual/faq/#faq-integer-ari... on the subject.

I'm still working through the paper—reading your group's Julia papers is always a great treat—thank you for this one and all of them. Looking at the definition of "type groundedness" it seemed to me that perhaps the definition should require that "every expression’s type depends only on the argument types" rather than only variables. I haven't gotten to the proof parts, but this was just a thing that struck me early on.

It's true that 1.0 feels really old at this point. We're also trying to improve messaging that people should generally not be using LTS unless they work in a really deeply risk-averse organization. Almost everyone should just use the latest release. That messaging should hopefully help make what the LTS release is less important. It just shouldn't matter to most people.

C# is certainly more flexible in this regard but I wouldn’t say they made the distinction between primitive and object invisible, they just allowed user-defined “primitives” in the sense that users can define C-style structs that have value semantics. The type system is still bifurcated between value types and reference types. That’s ugly but livable in a static language where you can just disallow mixing the two kinds of types. And it’s not just theoretically ugly: it means you cannot write generic code that works for both kinds of types. Or you can yolo it like C++ templates and just allow whatever by textual substitution—to hell with the semantic differences. But of course then you get all kinds of surprising differences in fundamental behavior based what kinds of types you try to apply your generics to. Which is exactly what happens with C++ templates.

What do you do in a dynamic language where no such separation can be enforced? Or in a static language where you just don’t want that kind of ugly anti-generic bifurcation of your type system? The classic static PL answer is to just make all objects have reference behavior. (Which is the approach Java takes with a pragmatic ugly exception of primitive types.) This is the approach that ML and its derivatives take, but which is why they’re unsuitable for numerical computing—in ML, Haskell, etc. objects have “uniform representation” which means that all objects are represented as pointers (integers are usually represented as special invalid pointers for efficiency), including floating point values. In other words an array of floats is represented in ML et al. as an array of pointers to individual heap-allocated, boxed floats. That makes them incompatible with BLAS, LAPACK, FFTW, and just generally makes float arrays inefficient.

So what does Julia do? Instead of having mutable value and reference types, which have observably different semantics, it has only reference types but allows—and even defaults to—immutable reference types. Why does this help? Because immutable reference types have all the performance and memory benefits of value types! Yet they still have reference-compatible language semantics, because there’s no way to distinguish reference from value semantics without mutation. In other words immutable structs can be references as far as the language semantics are concerned but values as far as the compiler and interop are concerned. And you can even recover efficient mutation whenever the compiler can see that you’re just replacing a value with a slightly modified copy—and compilers are great at that kind of optimization. So all you give up is the ability to mutate your value types—which you don’t even want to allow for must numeric types anyway—and which you can simulate by replacement with modification. This seems like a really good trade off and it’s a little surprising that more languages don’t make it.

That’s a really good question and I’m not sure I fully understand all the reasons. One big one is that Java intentionally has made interop with native libraries quite difficult. One cannot take a language seriously for numerical computing if you can’t easily call BLAS, LAPACK and FTTW just to start, and there’s a neverending supply of such efficient native libraries. Julia, on the other hand makes it easy to write efficient code in Julia but also makes it easy to call native libraries written in other languages. Easy interop with numerical libraries was pretty much the first design criterion.

There’s also some unfortunate choices Java made like standardizing one specific semantics for reproducible floating point code. That’s unfortunate because adjusting for native SIMD widths sacrifices reproducibility but improves both accuracy and speed. The only choice if you want perfect reproducibility on all hardware that Java supports is the worst performance model with the worst accuracy.

There’s also the fact that Java integers are 32-bit and Java arrays are limited to 2GB, which was reasonable when Java was designed, but is pretty limiting for modern numerical computing.

I also think that the JVM object model is quite limiting for numerical computing. They still don’t support value types, but value types are precisely what you want to represent efficient numerical values like complex numbers, or quaternions or rationals, and so on. Java forces all user-defined types to be heap-allocated reference types. Julia solves this by defaulting to immutable structures, which is exactly what you want for numerical values: the semantics are still referential (if you can’t mutate you can’t distinguish value semantics from reference semantics), you just can’t change values, which is exactly how you want numbers to behave (you don’t want to be able to change the value of 2).

Lack of value types in Java also makes memory management unnecessarily challenging. You can’t make a user-defined type with an efficient C-compatible array layout in Java. Because the objects are references, so the array is an array of pointers to individual heap-allocated objects. The ability to subtype classes forces that, but even with final classes, the ability to mutate objects also forces it, since pulling an object reference out of an array and modifying it is required to modify the object in the array (reference semantics), and that’s incompatible with the inline array layout.

And finally, this same lack of value types puts a LOT of pressure on the garbage collector.

Right, the question becomes: if you have `(a + b) + c` and one of the `+` operations allows re-association but the other one doesn't, then what happens? The problem being that associativity is a property that only makes sense for an entire tree of associable operations, not individual ones.

Local would be not using register flags at all and instead indicating with each operation whether you want flushing or not (and rounding mode, ideally). Some libms may clear and restore the control flags and some may not. Libm is just an example here and one where you're right that most of function calls that might need to avoid flushing subnormals to zero are expensive enough that clearing and restoring flags is an acceptable cost. However, that's not always the case—sometimes the operation in question is a few instructions and it may get inlined into some other code. It might be possible to handle this better at the compiler level while still using the MXCSR register, but if it is, LLVM certainly can't currently do that well.

The way Julia handles this is worth noting:

- `fma(a, b, c)` is exact but may be slow: it uses intrinsics if available and falls back to a slow software emulation when they're not

- `muladd(a, b, c)` uses the fastest possibly inexact implementation of `a*b + c` available, which is FMA intrinsics if available or just doing separate `*` and `+` operations if they're not

That gives the user control over what they need—precision or speed. If you're writing code that needs the extra precision, use the `fma` function but if you just want to compute `a*b + c` as fast as possible, with or without extra precision, then use `muladd`.

To be fair though, as noted further down in that thread, naive left-to-right summation is the worst case here since the tree of operations is as skewed as possible. I think that any other tree shape is better and compilers will tend to make the tree more balanced, not less, when they use associativity to enable SIMD optimizations. So while reassociation is theoretically arbitrarily bad, in practice it's probably mostly ok. Probably.

That's true, but the danger of flushing subnormals to zero is correspondingly worse because it's global CPU state and there's commonly used code that relies on not flushing subnormals to zero in order to work correctly, like `libm`. The example linked in the post is of a case where loading a shared library that had been compiled with `-Ofast` (which includes `-ffast-math`) broke a completely unrelated package because of this. Of course, the fact that CPU designers made this a global hardware flag is atrocious, but they did, so here we are.

> Maybe it would be nice if my compiler could automatically replace any inefficient loops with vectorized equivalents

The trouble is that a for loop is much more expressive than vectorized operations, so most for loops cannot be transformed into vectorized equivalents. The reason convincing people to write vectorized code works for performance is that you're constraining what they can express to a small set of operations that you already have fast code for (written in C). Instead of relying on compiler cleverness, this approach relies on human cleverness to express complex computations with that restricted set of vectorized primitives. Which is why it can feel like such a puzzle to write vectorized code that does what you want—because it is! So even if a compiler could spot some simple patterns and vectorize them for you, it would be incredibly brittle in the sense that as soon as you change the code just a little, it would immediately fall off a massive performance cliff.

I guess that's actually the second problem—the first problem is that there isn't any compiler in CPython to do this for you.

This feels a bit ironic given how much the for loop has been villainized in numerical computing, by none other than languages like Python—and Matlab and R—where for loops are so awfully slow that you can only get performance if you avoid them like the plague and write (sometimes awkward) vectorized code that pushes all the for loops down into some C library (or C++ or Fortran). Compared with, say, Julia, where people are encouraged to "just write yourself a dang for loop"—because it's not only effective and simple, but also fast. I guess what I'm saying is that it feels like even though Python may embrace the for loop as syntax, that affinity seems superficial at best, since if you care at all about performance then Python immediately rejects the for loop.

You say that as if fairness has anything to do with it. It doesn’t. Programming languages need to let people install packages on all the major platforms. End of story. Any approach that doesn’t make that practical and scalable is dead in the water regardless of fairness. Even just on Linuxes, there’s enough package manager variation to kill that idea for installing language-level packages with all the different system-level package managers. If you want to lobby all the Linux distros and Apple and Microsoft to adopt a common package installation standard, that would be great but until that happens, that’s just not a feasible way for languages to install packages.

In this vision of the world, how does one install Python packages on Windows or MacOS? Or even on Linux distros that don't use RPM or Apt?

Let's suppose that some language decides to use the system package manager to install packages. They add support for RPM. Then for Apt. Then for Portage. Then for Pacman. Then for Zypper. If they're not too exhausted at this point and have any spare time between trying to keep things working on every flavor of Linux around, they decide they want people to be able to use their language on Macs. So they add support for Homebrew. Except that some Mac users don't like Homebrew and want to use MacPorts. Some others use Fink. So the intrepid language devs add support for those two as well. Now our poor maintainers are supporting eight system package managers that all have their own quirks and oddities that makes keeping all this working a massive nightmare. And all of this work is not done just once, it has to be repeated for every single package that someone creates for the language.

Oh wait, but users still can't even install language packages on Windows! You know — the most popular OS of all. Oops.

Perhaps this sheds light on why no programming language does it this way. Languages have to allow package installation on not only your personal favorite distro of Linux, but also on vastly more popular OSes — Windows and MacOS — that don't even have standard package installers that they can use. The only scalable way to do this is to allow installation of the language and its internal package manager on each platform and then use a consistent internal system that is abstracted from the platform for installing language packages. Which, not coincidentally, is exactly what every programming language does. Python is a little odd in that it doesn't have a single blessed package manager, and that causes a lot of problems, but using the system package manager is not the solution.

The big problem in this post seems to be compiling via C (and the awkward GPU extensions thereof), which has lousy Float16 support. LLVM has adequate support for at least representing Float16s. Because of this (and a lot of work by a lot of people), Julia has pretty good support for Float16. If you're running on hardware with native Float16 support like a GPU, it works and is fast; if you're running on hardware without native Float16 support, operations are implemented by converting to Float32 and back, which is slow but gives the same results. So you can run the program either way and get the same results and it's fast if your hardware has native support. Same deal with BFloat16 [1], which is the native 16-bit floating point type on Google's TPUs.

[1] https://github.com/JuliaMath/BFloat16s.jl

Might make it easier to dispatch on both the object and the action? Just a wild guess.

Indeed. And in this case, the #3 contributor (and #1 in recent times, really) to Julia is also the current primary maintainer of libuv so it’s very unlikely that there would be any critical fixes that would be missed.

This particular example turns out to be a case where a feature was developed in Julia first and there’s some corner case on this poster’s Windows 10 setup that triggers a bug in the new feature.

Not to call anyone out but examples? Every time I look at packages I’m blown away by how much effort has gone into their docs. But people keep saying this kind of thing so there must be some packages they’re hitting that are under-documented.

The thing is no one is saying that starting a process every time isn't a valid way to work — it works great in most scripting languages. What people are saying is that since it's slow and annoying in Julia currently, you might want to consider a REPL-based workflow with Revise. If the response to that is "But I don't wanna use the REPL!!" then what can one do? Ok, don't use the REPL. But as everyone has agreed, it's going to be a bit annoying. If that's a dealbreaker for you then you may not want to use Julia just yet — which is totally ok!

That interaction unfortunately seems to often get characterized as "The Julia devs don't care about startup time! They insist on REPL-driven development and refuse to work on it." Never mind that improving startup time and package loading has been the number one priority of the compiler team since Julia 1.0 and that it's gotten about 10x faster. There's also ongoing work to make it even faster using system images.

I was curious about that too. Many other people use Windows 10 and this is the first time this issue has come up, so there must be something else unusual going on here. If no one could install packages on Windows 10, we'd have heard about it.

Always good to attract experts from other languages. I was mainly addressing the claim that Julia core devs like to crap on other languages, not trying to claim that we're experts in every language — obviously we're not. But saying that we're uninterested in other languages or unwilling to learn from them is simply untrue — we're a bunch of huge programming language nerds and we love programming languages, even some of the notoriously flawed ones like C++, Matlab and Perl.

Quite a bit: several of the Julia compiler team are LLVM maintainers and contributors. LLVM has started to track compile time, so that's gotten a bit better with recent releases. Making noise about compile time was mainly to get the LLVM project to start paying attention to compile time, which they now have, so mission accomplished — as long as they stop completely allowing latency to degenerate, it's ok. LLVM is never going to be fast at compiling code (and if we used something else that was as good at producing fast code, it would almost certainly also be slow), so avoiding putting LLVM in the fast path is always going to be necessary to make more latency improvements.

The announcement post for Julia was literally about all the things we love in other languages:

https://julialang.org/blog/2012/02/why-we-created-julia/

It’s not really a problem with dynamism, it’s more about the fact that Julia’s compilation model is similar to C++ with templates but at runtime (and with first class parametric types instead of textual substitution, but the compilation and specialization story is similar). Just as separate compilation isn’t possible for C++ template code, it’s a challenge for Julia as well.

Using LLVM is not exactly an indelible architectural decision — switching to a bespoke JIT would be entirely possible, it’s just not worth it yet. Moreover, if you want to generate world-class high speed code, you really need something like LLVM. So yes, it’s a bit of a drag but it’s also a massive enabler of critical performance. That’s the real reason the project continues to use it.

There are a number of approaches to reducing latency further, which are being pursued. The most promising current avenue is better tooling around system images (compiled executables with additional functionality beyond the core language pre-loaded): generating them faster, generating them incrementally (load one, add a few things and produce another), and generating them automatically for the set of packages in the current active project (not feasible until the first two are done). That will give an experience comparable to compiled languages.