Julia 1.9 precompilation will be a turning point | Hacker News Reader

Julia 1.9 precompilation will be a turning point

twitter.com

76 pointsptype14 comments

Over the years I’ve kept coming back to Julia, like what I see, but ultimately end up with a deadline that has me reaching for R or Python. My latest foray has convinced me that the ecosystem is ready for me to double down and make it my language of first recourse for data science related tasks this year.

Precompilation sounds like it improves further on an annoyance (time-to-first-plot problem) that was already no longer much of an annoyance in Julia 1.8x.

Just to manage expectations: As far as I understand, v1.9 doesn't by itself solve TTFP/precompilation. The tweet seems slightly over-enthusiastic. But it puts tools to solve it in the hands of package developers.

In that sense, one can perhaps say it is a turning point.

While that is an accurate statement, my go-to package where compile time and startup time was a huge hurdle, Plots.jl, just works instantaneously now. That is the case for many packages already, and I suspect that after 1.9 releases, it will quickly become more commonplace.

Ah, thank you for that context. Assuming this mechanism doesn’t have any major tradeoffs, I think it seems likely that popular packages will make use of it?

Two tradeoffs to be aware of:

1. Slightly longer compile times at package install time, but that is ok since it is a one time cost.

2. During package development, building a pkgimage every time will lead to longer compile times because of native code generation. That is easily disabled during development with `julia --pkgimages=no`.

The popular packages are already preparing for this.

It's very likely. I've only occasionally used in julia for a couple of years and I've been doing various hacks to improve load/compile times for my private packages pretty much since the beginning, so I imagine more experienced programmers would jump at it.

Yes, I think so (and many already have)

As package developers we need to optimize our packages for 1.9. It's quite a task but I am excited what's ahead in Julia. Matlab(is not open-source, R is slow and not really a general purpose language, Python is great but same issue 2-lang problem...why should I implement CUDA in C++ or Numpy in C. I want to be able to modify lower back-end code but with Python it's not possible. Julia fixes all of these problems and I am quite happy I invested my time in Julia. Present/Future is bright :)

Thankfully my GPU programming is shadet related, thus I don't have to deal with all the compute issues, however a good decision of CUDA from early on was to have PTX.

GPGPU needs more languages that make it easier to do compute with feeling like doing Assembly.

Basically the productivity that can be understood by reading stuff about StarLisp and the connection machine.

Julia seems a nice addition to this idea.

I don’t understand how this wasn’t already implemented, it seems like such low hanging fruit

The Julia compilation model is fairly complex, due to the Julia user's wish for dynamic capabilities combined with common features in statically typed languages.

Although it may appear so, it was not low hanging fruit - it took a lot of effort building incrementally over several releases by a number of contributors.

In retrospect, how essential was it that Julia went the dynamic route in the first place (as opposed to taking a Haskell-like approach to inferred static typing)?

It's a pretty major difference. Julia lets you write code that the type system doesn't understand, while in a static functional languages you generally have to explain a decent portion of category theory before someone can start writing code that passes around functions. (For a simple example, what is the type of `+`, and does that type allow you to make Int+Float=Float?)

To elaborate on the sibling comment: Julia has (1) multimethods / multiple dispatch capabilities, (2) you can dynamically define new types and methods at runtime, (3) everything is aggressively devirtualized/compiled to machine code when possible. That requires the capability of discarding and recompiling code that has already been compiled and cached when a new method is defined at runtime. Worse even, just importing a new library might invalidate some of the compiled cached code. For me, capabilities 1,2,and 3 make the cost of a complicated compilation model worth it. But it also means it takes a lot of time to improve the compiler (this type of better caching was not at all a low hanging fruit).

Reply on news.ycombinator.com