Hacker News Reader

Indeed all the DockYard pieces announced are nice, but the posts are puff pieces at best.

From what I understood, Elixir/OTP are there, but instead of compiling to BEAM bytecode, they compile for WASI (targeting whatever can run WebAssembly). It is not HiPE, nor JIT, there will not be any bytecode. Only AOT compiled native code, except targeting WASI.

Generally BEAM is understood to be slower of the big runtimes (compared against Java, CLR, Go and often V8/Javascript). FireFly claims to be faster, and smaller in compiled form than current interation compiled against BEAM.

How that plays with actor model and preemptively switched lightweight processes is a mystery to me too.

I'd really appreciate something with more technical sauce but human explanation. Currently all we have is PR puffs like this and source code (at least for the LiveView native, haven't tried looking for this).

All that said, this does indicate there is good activity and desires to get Elixir working on stuff other than its current strong suit. This usually means the language and ecosystem is growing. It could be Baader-Meinhof phenomenon, but I started learning Elixir couple of weeks ago, starting with LiveView tomorrow, and there is nice stream of posts coming for it. Exciting times :)

> Only AOT compiled native code, except targeting WASI.

More precisely, a number of targets are supported, WASI would be just one.

> Generally BEAM is understood to be slower of the big runtimes (compared against Java, CLR, Go and often V8/Javascript). FireFly claims to be faster, and smaller in compiled form than current interation compiled against BEAM.

The point is that by placing some restrictions on what is possible at runtime (specifically by removing the possibility of hot code loading), we can do whole program analysis and thereby do much more aggressive forms of optimization and dead code elimination across compiled applications _and_ the runtime they link to. It isn't guaranteed that such programs would be faster (though I suspect in some cases they would be), but they almost certainly should be smaller, which is important for Wasm, and other constrained targets.

> How that plays with actor model and preemptively switched lightweight processes is a mystery to me too.

It makes no difference, you can implement all of that with identical semantics from the perspective of the developer, the strategy used for compilation is orthogonal to those features, though naturally the implementation details are tightly integrated.

> It isn't guaranteed that such programs would be faster (though I suspect in some cases they would be), but they almost certainly should be smaller, which is important for Wasm, and other constrained targets.

Smaller in code size, maybe (though that possibly doesn't matter given that code is pretty compressible), but possibly less cache-coherent! One thing that's interesting about bytecode interpreters like BEAM, the JVM, etc., is that since all the actual native-ISA code is just the same small set of instruction impls being jumped to over and over, the interpreter can stay entirely hot in L2 or even L1 cache at all times; with the program bytecode being executed through, while less hot, being more concise, and therefore also being "hotter per byte" since code that does more per op will take longer to fully run through before it needs to be evicted in favor of something else.

This has been a consideration for decades—it's why programs compiled to Pascal p-code tended to be faster than programs compiled natively for the low-level (mostly ALUless) host instruction sets of the time.

Smaller code size was the goal; and BEAM bytecode adds up quick, even compressed. For example, the BEAM files for _just_ Ecto come out to about 1.2M in an uncompressed tarball, but in a gzipped tarball at maximum compression is still 799K. In my experience, about the smallest release for the average application in terms of BEAM bytecode is about 30M (uncompressed), across the standard library, dependencies, and your own code. Virtually none of that can be dead code eliminated, because the BEAM has to treat it all as potentially reachable. Shipping that much to a browser (even compressed) is just not viable.

Your point about instructions remaining hot in the cache might very well be true more often than not, but is highly sensitive to the application in question. The core interpreter opcode impls might all fit in cache at the same time (though I doubt even that with the BEAM due to how many there are), but any call to BIFs/NIFs is likely to cause evictions. It still might do better than natively-compiled code overall in that specific sense, but I would be hesitant to state any generalities about it when considered as a whole with all of the many other factors that play in to overall performance.

In any case, Firefly wasn't about building a faster BEAM, but about bringing BEAM languages to the browser (or really any Wasm host). While targeting standard server/desktop architectures was something we also wanted to support, particularly for writing CLI tools and such, we expect the BEAM will always be the first choice for people deploying to those systems. If we can build something that is faster than the BEAM in some cases due to the tradeoffs we make, that's great, but it isn't an explicit goal.

> For example, the BEAM files for _just_ Ecto come out to about 1.2M in an uncompressed tarball

Did you try stripping out debugging chunks? IIRC every Elixir module embeds its own source code by default.

Thank you! That makes quite a few things clear.

As a naïve question, how does SBCL (Common Lisp compiler) generate such a wicked fast code? Other than actor model, it is also dynamically typed and hot code loading is one of the headline features there.

When I say hot code loading, I’m really specifically referring to how that works in the BEAM, which is a more sophisticated mechanism than simply compiling/generating code on the fly. The biggest problem though is that it prevents most forms of dead code elimination. It also means that you can never assume anything about how a function will be called, because at any point new code could be loaded that calls it differently. You can still optimize such code with a tracing JIT, but in Wasm (at least currently) that’s not even an option.

Without knowing too much detail about SBCL specifically, I suspect that they use a combination of clever static analysis and specialization to unlock a lot of that speed. That way functions can be specialized/inlined where beneficial, but new code can safely call the unoptimized versions when loaded at runtime, but even hot loaded code could be specialized with a JIT on hand. The big reason we made the trade off with hot code loading though is due to the restrictions that Wasm imposes - there’s no particular reason we couldn’t support it otherwise. In general it is rarely used in production BEAM apps in my experience, so from my perspective it seemed like an opportunity to stop paying for a feature unused and gain something in return.

Thank you for detailed reply here and elsewhere in the thread. That probably took as much time as writing a new blog post, so it is really appreciated.

As mentioned before, as a newbie student of Elixir this is all very exciting. Please keep up the good work!

If you are looking for more substantial technical information, they are found in the ElixirConf ‘19 keynote speech and demo when Lumen was announced.

https://youtu.be/uMgTIlgYB-U

Agreed, these posts from Dockyard have been really low quality. Dockyard makes some awesome stuff generally, so I'm surprised at how shallow and marketing-speak these have been.

That said I'm really excited for this and the other projects. Looking forward to some more technical dives.

Speaking as the lead on the project, this is partially due to this weekend being ElixirConf, so things are hectic, but you can also blame me, as I probably should have written this post, but didn’t make the time as I was pretty heads down on the lead up to the conference.

I’ll make sure to do a follow up post in the near future that is more in depth

Thank you for all the follow up posts on this thread... It strikes me they're probably taking up as much, if not more, time just in the clarifications!

They make up for it in their ElixirConf keynotes and presentations.

I'd love to watch it, but the recordings are not available yet and 250$ virtual.attendance was way too much for personal interest language.

They are published free after the conference. The 2019 keynote with substantial technical information on Lumen is available for free on youtube.

> What does this mean? Erlang/Elixir are AOT-compiled languages..

With the BEAM compiler today (and by BEAM compiler, they are referring to the compiler provided as part of the standard library shipped with the BEAM, it is just a way to clarify which Erlang compiler is referred to), Erlang sources are partially AOT-compiled to BEAM bytecode, then the code loader does additional compilation steps at runtime. Firefly AOT compiles Erlang to native code directly, and does not use a virtual machine at runtime.

> So is the key benefit here that the compilation is faster because it's not running on the BEAM (good for e.g. CI); or is the key benefit that the resulting executable is faster because it's not running on the BEAM?

There are a few benefits we hope to provide using this approach:

1. Compilation can be faster because the compiler is implemented in Rust, rather than implemented in Erlang and running on the BEAM. 2. We impose a restriction that hot code loading is not permitted, so we are able to do forms of optimization that the BEAM cannot do, as a result of having to pessimize in the presence of hot code loading. 3. Related to above, we can do whole program analysis, including optimizations that take advantage of unboxing terms and working with more machine-friendly types. 4. We can produce a single statically-linked executable that is as small as possible, in part due to being able to do more aggressive and precise dead-code elimination across both the application being compiled and the runtime it links to, because we know what must be available at runtime. On the other hand, if one were to ship the BEAM itself to run in browsers via Wasm (if it was modified in such a way as to be possible), you'd also need to ship large amounts of bytecode for most applications, regardless of whether much of that bytecode is actually needed at runtime. Deployment has always been a pain point of the BEAM (and I'm speaking as someone who built the primary release tools for the Elixir ecosystem), Firefly aims to make deployment as simple as Go/Rust/etc. 5. I'd like to be able to take advantage of the fact that we use MLIR behind the scenes to explore using Firefly as a natural pairing with Elixir applications using Nx, by being able to more seamlessly integrate regular Elixir code with Nx-managed functions.

> How does a non-bytecode version of Erlang/Elixir abstract-machine semantics, achieve these same guarantees? Is there an explicit reduction-counter being carried around in the emitted native code?

In short, yes, we implement preemptive-scheduling the same way the BEAM does, using compiler-injected yield points based on a few criteria (a reduction counter is just one, some others are garbage collection and blocking I/O).

> And also, there is no mention of disadvantages/constraints of using this system. It's pretty clear that you wouldn't be able to do hot reloading or dynamic trace-point insertion without the BEAM there to intermediate it. That's fine for some use-cases, but they should explicitly mention the trade-offs and target audience.

The readme of the project certainly does this, but I agree it would have been good to include in the blog post. In any case, Firefly is still in early stages, so this isn't something anyone is using today. When we reach the point where we feel it is production ready, I can assure you I will be writing up a very detailed analysis of what it is ideally suited for, and what it is not - like you, I feel it is critically important to be clear about that.

> Compilation can be faster because the compiler is implemented in Rust, rather than implemented in Erlang and running on the BEAM.

If this is a full-blown Erlang/Elixir (or only Elixir?) compiler, rather than a "BEAM IR to native" compiler, that sure sounds like a lot of independent work to maintain, especially without there being any sort of spec/standard defining what the behavior of such a compiler should be separate from what the reference compiler does, and what the resulting abstract machine should do separate from what the reference VM does. How do you plan on ensuring your compiler tracks updates introduced by major Erlang releases? Will there be features introduced in the Erlang runtime (I'm thinking things like Erlang 21's `atomics`) that won't be immediately available for FireFly-compiled programs, where FireFly will just choke on these?

> We impose a restriction that hot code loading is not permitted, so we are able to do forms of optimization that the BEAM cannot do, as a result of having to pessimize in the presence of hot code loading.

Hot code loading happens in places other than just relups, though, no? There are some Erlang/Elixir libraries which do crazy things at runtime — for example, protocol serialization libraries which, at runtime, discover remote schemas; dynamically generate code to ser/des values for those schema; compile that code to a module in memory; and then load the resulting module. And this is not discouraged/disincentivized in the Erlang ecosystem; the compiler infrastructure is usually considered to be "part of the standard library," for any application to use freely at runtime. So even if I don't do any of this in my own project, I would be worried that some transitive dependency of my project would be secretly doing this.

Even if you have no plans to support hot code loading in the "remote calls can jump into the new version of a module" sense, do you think it would make sense for FireFly to ever support "module compilation+loading at runtime"? Maybe with the resulting modules being native DLLs?

---

Tangent to that — and I know that this was probably nowhere on your mind when you were working on the project, but something interesting to consider: how hard would it be to convince the FireFly compiler to emit a C-ABI library that could be loaded into the BEAM as a NIF?

I ask, because this would be a really interesting way of achieving the same sorts of speedups what HiPE does/did — but more explicitly, on a higher unit level, and (probably) better.

I have a similar/related question as well: Will Firefly support most/all elixir libraries, or is it expected/known that apps will have to avoid some common APIs in order to ensure they avoid illegal calls?

The goal is to support all libraries, but because there are some out there that dynamically compile/load code at runtime, despite it being a bad idea, there will necessarily be some libraries that are not supported, at least in the near term. Likewise, you might also want to compile a library for say, Wasm, that uses a NIF which lacks support for that target, which wouldn't work; but this is already a restriction with the BEAM today with NIFs (i.e. if they don't support a particular target, then the library won't work).

But as long as a library only uses code loading APIs at compile-time, and not runtime, it should be supported just fine. To be clear, we only plan to raise runtime errors in those cases, as if the call failed normally, and not prevent compilation just because a call to one of those APIs exists in the code, though we may choose to emit diagnostic warnings for it, that remains to be seen.

> How do you plan on ensuring your compiler tracks updates introduced by major Erlang releases? Will there be features introduced in the Erlang runtime (I'm thinking things like Erlang 21's `atomics`) that won't be immediately available for FireFly-compiled programs, where FireFly will just choke on these?

We would plan to track Firefly releases against specific mainline Erlang/OTP releases, so for example, we would explicitly state that Firefly v1 tracks Erlang/OTP 25, v2 tracks 26, and so on.

Most changes implemented in each release are implemented in the Erlang-based standard library, so we would get most of that for free (we aren't planning to maintain a completely separate fork of OTP, only the parts that we need to implement ourselves, primarily stuff that is already separate and provided in the preloaded modules shipped with ERTS). What remains is typically small enough that we should be able to maintain parity pretty closely, though naturally there may be delays depending on the scope/effort involved. Obviously we hope to grow enough community around the project that this is a non-issue, but the project has enough financial backing at this point to handle this in the near term. I also hope to make alternative runtimes/implementations of Erlang/OTP something that the core team takes into account via the EEF, ideally resulting in some kind of technical spec around semantics of the language; in the near term we have to rely on the OTP test suite, various papers that have been produced over the years, and a whole lot of digging through the ERTS code itself.

> Hot code loading happens in places other than just relups, though, no?

Yes, of course, and obviously that means some libraries won't be compilable with Firefly. In my experience though, the majority of production apps are not and should not be doing dynamic code generation/loading at runtime; that's certainly something I would raise in a code review if I saw it. Naturally there are going to be teams out there that _are_ doing so, or use tools/libraries that do so, but I'm fine with this being a reason why you would choose the BEAM instead.

> Even if you have no plans to support hot code loading in the "remote calls can jump into the new version of a module" sense, do you think it would make sense for FireFly to ever support "module compilation+loading at runtime"? Maybe with the resulting modules being native DLLs?

We do support dynamic loading of libraries containing new modules/functions, and I think it is certainly possible for us to provide a JIT on supported platforms in the future to do the kind of unbounded code generation/loading that the BEAM supports, but it is not a priority by any means. As I've stated previously, it is rare that I've seen that kind of thing abused in production applications, and I don't see it as a major selling point of the platform (or a blocker for Firefly). It is a tradeoff though, and that does mean there will be things you just can't do with Firefly - but I think that's an important property of alternative implementations; ideally you want them to be tailored towards different use cases, otherwise there is little reason to have multiple implementations in the first place.

> Tangent to that — and I know that this was probably nowhere on your mind when you were working on the project, but something interesting to consider: how hard would it be to convince the FireFly compiler to emit a C-ABI library that could be loaded into the BEAM as a NIF?

You'd be surprised what things I've considered while working on this project ;). I think it is certainly possible, and I'd like to support it, especially since I think it might be a great way to use Firefly in a traditional BEAM deployment for things that, as you mentioned, one would have previously considered using HiPE for. I have yet to investigate just how tight that integration could be though.

Thank you for elaborating!

I think that Erlang/Elixir could make amazing TUI apps if they were easier to distribute. The ability to push work to the edges of the system while keeping the UI responsive would be awesome. It sounds like static binaries provided by Firefly would make this a viable option.

Likewise. I still mainly use Ruby or Go for writing CLIs, but I would much, much rather use Elixir. escript is nice but still requires the target to have erlang installed. Being able to produce a statically-linked binary would be enough to just use elixir only, which is my dream.

This is one of the things I'm personally excited about using Firefly for, since I first started working professionally with Erlang/Elixir, I wanted the ability to use it for CLIs. You can be sure it will be a use case well supported :)

While I'd be fascinated to hear the answers to all your questions, I would guess maybe 2% of people would even understand them. I wouldn't expect a post like this to get anywhere near that technical.