With the BEAM compiler today (and by BEAM compiler, they are referring to the compiler provided as part of the standard library shipped with the BEAM, it is just a way to clarify which Erlang compiler is referred to), Erlang sources are partially AOT-compiled to BEAM bytecode, then the code loader does additional compilation steps at runtime. Firefly AOT compiles Erlang to native code directly, and does not use a virtual machine at runtime.
> So is the key benefit here that the compilation is faster because it's not running on the BEAM (good for e.g. CI); or is the key benefit that the resulting executable is faster because it's not running on the BEAM?
There are a few benefits we hope to provide using this approach:
1. Compilation can be faster because the compiler is implemented in Rust, rather than implemented in Erlang and running on the BEAM. 2. We impose a restriction that hot code loading is not permitted, so we are able to do forms of optimization that the BEAM cannot do, as a result of having to pessimize in the presence of hot code loading. 3. Related to above, we can do whole program analysis, including optimizations that take advantage of unboxing terms and working with more machine-friendly types. 4. We can produce a single statically-linked executable that is as small as possible, in part due to being able to do more aggressive and precise dead-code elimination across both the application being compiled and the runtime it links to, because we know what must be available at runtime. On the other hand, if one were to ship the BEAM itself to run in browsers via Wasm (if it was modified in such a way as to be possible), you'd also need to ship large amounts of bytecode for most applications, regardless of whether much of that bytecode is actually needed at runtime. Deployment has always been a pain point of the BEAM (and I'm speaking as someone who built the primary release tools for the Elixir ecosystem), Firefly aims to make deployment as simple as Go/Rust/etc. 5. I'd like to be able to take advantage of the fact that we use MLIR behind the scenes to explore using Firefly as a natural pairing with Elixir applications using Nx, by being able to more seamlessly integrate regular Elixir code with Nx-managed functions.
> How does a non-bytecode version of Erlang/Elixir abstract-machine semantics, achieve these same guarantees? Is there an explicit reduction-counter being carried around in the emitted native code?
In short, yes, we implement preemptive-scheduling the same way the BEAM does, using compiler-injected yield points based on a few criteria (a reduction counter is just one, some others are garbage collection and blocking I/O).
> And also, there is no mention of disadvantages/constraints of using this system. It's pretty clear that you wouldn't be able to do hot reloading or dynamic trace-point insertion without the BEAM there to intermediate it. That's fine for some use-cases, but they should explicitly mention the trade-offs and target audience.
The readme of the project certainly does this, but I agree it would have been good to include in the blog post. In any case, Firefly is still in early stages, so this isn't something anyone is using today. When we reach the point where we feel it is production ready, I can assure you I will be writing up a very detailed analysis of what it is ideally suited for, and what it is not - like you, I feel it is critically important to be clear about that.