Hacker News Reader

Comment by jakobnissen | Hacker News Reader

It's a fundamental problem of having a dynamic, compiled language. It's not possible to actually solve. Think of it similar to how C++ and Rust needs to compile its source code before running it. No-one talks about this as a problem to solve.

The problem with simply compiling dependencies to static binaries is that all Julia code is allowed to redefine other Julia code. So package X can load package Y, then define a method that causes package Y to change behaviour, and thus needs to be recompiled.

This is not unintentional, by the way. Having packages able to use each other's code is critical for having the ecosystem be "composable", that is, being able to make two different packages work effectively together. For example, I might take a type from package X and throw it into a function from package Y, and it works.

View on HN

I don't think this is an inherent problem of having a compiled dynamic (in other words, JITted) language.

Things like Javascript (V8) and Lua (LuaJIT) manages to have fast startup times while having exceptional performance in hot paths - this is because they have a fast bytecode interpreter that executes the script first while the actual compilation is taking place. Unfortunately Julia in its current state, doesn't have a fallback interpreter to execute its bytecode/IR (which is similar to LLVM IR). And LLVM IR isn't really suited for fast execution in interpreters - it's moreso designed as an intermediate language for a heavyweight compiler than a bytecode for a dynamic language.

Maybe some heroic figure would come out of the fog and suddenly brings a whole new bytecode representation and a fast interpreter for Julia, but that would be quite a project..

Julia does have a minimal compilation path with an interpreter. You can even configure this on a per-module basis, which I believe some of the plotting packages do to reduce latency. There is even a JIT-style dynamic compiler which works similarly to the VMs you listed: https://github.com/tisztamo/Catwalk.jl/.

IMO, the bigger issue is one of predictability and control. Some users may not care about latency at all, whereas others have it as a primary concern. JS and related runtimes don't give you much control over when optimization and are thus black boxes, whereas Julia has known semantics around it. I think fine-grained tools to externally control optimization behaviour for certain modules (in addition to the current global CLI options and per-package opt-ins) would go a long way towards addressing this.

You're right, done some more research and there seems to be an interpreter in the compiler: https://github.com/JuliaDebug/JuliaInterpreter.jl. It's only enabled by explicitly adding annotations in your code, and is mainly used for the internal debugger, but it's still there.

Still, it still seems to try executing the internal SSA IR in its raw form (which is more geared towards compiling rather than dynamic execution in a VM). I was talking more towards a conventional bytecode interpreter (which you can optimize the hell out of it like LuaJIT did). A bytecode format that is carefully designed for fast execution (in either a stack-based or register-based VM) would be much better for interpreters, but I'm not sure if Julia's language semantics / object model can allow it. Maybe some intelligent people out there can make the whole thing work, is what I was trying to say.

The naming is unfortunate, but JuliaInterpreter is not the built-in one but a separate package for use in external tooling. The built-in one can be run globally via the --compile=min CLI flag. Likewise, you can also pass -O0 to -O3 to configure the level of optimization (which, predictably, affects latency).

As for IR representation, I'm not aware of any limitations of the IR (remember, there are multiple levels) over a LuaJIT-style bytecode for interpretation performance. After all, the Futamura projections tell us that a compiler is really an interpreter that's undergone some partial application. Of course, that's a theoretical correspondence that has little bearing on real-world performance, but I don't think you can confidently say that Julia's lowered or unlowered IR forms are fundamentally bad for fast interpretation.

You can set optimization per module with `Base.Experimental.@optlevel`, though I'm not finding any documentation for it. Could swear it was in a release note.

Yep!

  help?> Base.Experimental.@optlevel
    Experimental.@optlevel n::Int

    Set the optimization level (equivalent to the -O command line argument) for code in
    the current module. Submodules inherit the setting of their parent module.

    Supported values are 0, 1, 2, and 3.

    The effective optimization level is the minimum of that specified on the command line
    and in per-module settings.

Inside Julia compiler/runtime there is an interpreter, because Julia uses a heuristic to determine whether to compile or interpret a function. There is also interpreter code in Julia debugger. I don't know how full featured they are, but one does not have to start from scratch.

On the other hand, implementing a tracing JIT for Julia is going to be such a big task, I am not sure how much help existing interpreters are going to be. At the very least there needs to be a new GC, which necessitates changes everywhere except the parser. LLVM integration may also prove awkward for a tracing JIT.

> It's a fundamental problem of having a dynamic, compiled language. It's not possible to actually solve.

There are plenty of Common Lisp implementations that allow you to redefine functions or add methods to multi-dispatch functions without requiring you to recompile every use-site. There are other trade-offs here, but it’s not “impossible” to precompile and optimize code in a dynamically-typed language.

It’s not really a problem with dynamism, it’s more about the fact that Julia’s compilation model is similar to C++ with templates but at runtime (and with first class parametric types instead of textual substitution, but the compilation and specialization story is similar). Just as separate compilation isn’t possible for C++ template code, it’s a challenge for Julia as well.

> No-one talks about this as a problem to solve.

Maybe you just don't read the right places because people definitely talks about this as a problem to solve. For example, Julia already stores e.g. inferred code in the precompile files. There isn't a huge leap to also stored actual compiled code from that. Just because it is possible to invalidate code and cause recompilation doesn't mean that it isn't worth storing ahead of time compiled code in the 99.9% cases where invalidation doesn't happen.

But this is not solving the problem, it's just reducing it. My analogy with compilation of C++ is that noone SHOULD expect it to be "solved", because it's a natural consequence of how the compilation works. They may improve e.g. incremental compilation to reduce compile times, but it's just not a problem that can be "solved". Similarly with Julia and latency.

I don't see a difference between solving a problem and reducing it to not be an issue (or problem) anymore.

I disagree that it cannot be obviated in Julia. Can be solved with one or more of static tooling (ie proving there's no overwriting), tiered compilation with a faster interpreter and sealed modules. All things jeff et al have been discussing

You compare it to Rust compile times, but Julia's are much worse than Rust's. I say this as someone who has used both languages quite a lot.

C++ and Rust don't need to compile dependencies between program code modifications. Compiling a Rust program is faster, anecdotally, than running an equivalent Julia one. At least for the numerical and plotting uses I've tried.

Reply on news.ycombinator.com