Hacker News Reader

Comment by crabbygrabby | Hacker News Reader

Yea but keep in mind anytime you get a bug your session goes away so you can end up eating hours from a day precompiling. Not worth it with heavy pkgs imo. Wish they had incremental compilation like rust because most compiled Lang's are noticeably faster than Julia precompilation ime.

View on HN

> anytime you get a bug your session goes away

This only happens for Segmentation fault type crashes which kill the entire Julia process, which shouldn't be common at all. Can you describe when you experience these issues?

It's super easy to get ooms and seg. Faults in Julia. For a while there this past year you couldn't even Ctrl c to stop the Julia process. It's rough for real work imo. Fine for research.

I don't remember C-c being broken in the past year. Could it have just been a specific program you were running with a tight loop that didn't have any yield points? If so, that's not really unique to Julia.

I'd also be interested in your workload that was generating lots of seg faults (oom makes some sense if working with large data since Julia's runtime does add an unfortunate amount of memory overhead.

Check the version summaries. Think it was Julia 171 or something. There's lots of bugs like that that crop up every other release.

Segfaults happen all the time with FFI. But yea OOM is a killer. Julia runtime guzzles RAM, but pkg add any of the sciml stuff and watch your RAM explode. Doesn't take much data to lose a half hour of your life installing a package...

The RAM usage is because precompilation runs in parallel. If you have like 16 threads going then yes it'll parallelize that over 16 times. But we have never seen a half hour package install, can you share the info to reproduce it?

On my decade old laptop `using Plots` (precompilation) takes a couple of minutes. Not half an hour but it may feel like that if someone is used to Python (or R) where imports are instantaneous. Though I think GP meant takes that much due to OOMs which may result in frozen or slow system.

A couple of minutes is expected. Even R and Python packages have to run the build processes for the associated C and Fortran codes, and those take similar time (or more for many packages). However, if a Julia package is precompiling for more than half an hour, that's not expected and I'd like to see a reproducer for this so we can fix it.

See my above comment for clarification

It's not a half hour package install. It's a half hour of loading everything back in to get to where you were because the runtime dropped sorry for the lack of clarity. But yea, it's real easy for pkg installs to oom people.

Julia devs seem to work on improving precompilation times every version. That said if a project or workflow revolves around some packages, a way to skip this step is to create a sysimage, essentially a saved Julia session, that includes those packages. PackageCompiler significantly simplifies the process.

Reply on news.ycombinator.com