We are always working on streamlining PackageCompiler.jl and building binaries. Bug reports always appreciated.
I'm curious to understand why the 130MB is prohibitive. Are you looking at embedded or resource constrained environments?
I'm curious to understand why the 130MB is prohibitive. Are you looking at embedded or resource constrained environments?
I was just trying to do a quick test app the other day to see if I could build a Hello World app, and (this may have been my fault) but I hit the threshold of cognitive overhead before deciding to revisit later. This isn’t a criticism so much as a recognition that compiling binaries may take a bit more care / attention / learning in Julia than something like Go, Rust, or even Racket.
As for binary sizes, for some serverless deployments smaller binaries can open up a little easier path.
I’ll try and take another run at PackageCompiler and give it a more fair shake. If in that process I find anything constructive to report or submit documentation PRs, I will do so.
Thank you for your work on Julia. Julia was on my radar a few years ago and then dropped off again. I’m excited to see where it has come in that time.
This pattern can be seen in several places and frameworks like Spark & company, and it's also pretty much the foundational idea behind processing data with Joyent's Manta[0]. It always pretty much boils down to "serialize a function, and send it somewhere, to run on the data locally".
Some of these frameworks do it only through native functions (Spark runs serialized JVM procedures, and for Python functions I believe it shells out to an actualy Python interpretes), others are completely agnostic to this and work with binaries (Manta).
It'd be pretty hard to do, and have a lot of overhead, if all of those "functions" that I'm shuffling around are, at the lower bound, 130MB in size.
I understand that a lot of this can be done directly at the Julia layer for Julia things (much like Spark does with JVM things), but it kinda limits the applications of this nonetheless.
https://docs.julialang.org/en/v1/manual/distributed-computin...
However, this is not related to the system image (sys.so) that is used to start the Julia process, which has a compiled version of base, stdlibs and whatever else you choose to build in.
This is a nice alternative to the AOT compilation offered by the likes of PackageCompiler.jl
Hopefully we'll have more / better AOT compilation and deployment in the near future. There's a lot of rumblings of exciting things starting in Julia AOT compilation, and I think it'd make a killer combination with our package manager's binary artifact deployment story.
In practice, it may often inline more aggressively as there aren't any boundaries that will prevent inlining.
If you write function `foo` calling `buz` from package `Buz.jl`, which calls `bar` from package `Bar.jl`, `bar` can be inlined into `foo`.
You can think of it as always have `-lto`, because the source of the Julia functions you're calling will always be available to the compiler and thus subject to (potentially aggressive) optimization.