Lightbeam: A Highly-Optimising Streaming WebAssembly Compiler (2019) [video] | Hacker News Reader

Lightbeam: A Highly-Optimising Streaming WebAssembly Compiler (2019) [video]

youtube.com

99 pointsnonsince15 comments

Nowdays merged into wasmtime: https://github.com/bytecodealliance/wasmtime

Ok, but is it cool and why? I was hoping for more comments here. :)

Do you know how wasmtime comapres to wasmer?

Hi, I'm Syrus (founder of Wasmer [0]).

First things first: we have the deepest appreciation for Jack and the people behind the effort at lightbeam. They did a lot of amazing work on that space that inspired our work in the Singlepass compiler (similar design principles as lightbeam).

On that sense, Wasmer is built to support any kind of compiler backend. We currently support Singlepass (streaming compiler), Cranelift and LLVM (for super optimal environments). [1] We also support ARM [2] chipsets and sponsored the work on the dynasm Rust project to enable it.

Wasmer runtime is also highly pluggable via middlewares. For example, we added ultra-optimal "gas metering" as a part of it.

If you are curious about runtime speed (tldr: we are ahead ;) ), here are some other articles [3] [4]

Hope this is useful!

[0]: https://github.com/wasmerio/wasmer

[1]: https://medium.com/wasmer/a-webassembly-compiler-tale-9ef37a...

[2]: https://medium.com/wasmer/running-webassembly-on-arm-7d365ed...

[3]: https://00f.net/2019/10/22/updated-webassembly-benchmark/

[4]: https://medium.com/@leonardas103/experimenting-with-webassem...

I was looking that up myself and came across a reddit thread [1] with a Wasmer employee seemingly equitably laying out the general differences.

This, however, elicited a response from a Mozilla employee claiming it wasn’t telling “the whole story.”

Does anyone know what the differences in perspective are here?

[1]: https://www.reddit.com/r/rust/comments/cx4nj3/comment/eykkie...

wasmer used a large amount of wasmtime code without proper attribution. I'm not sure this is a problem anymore but habits die hard and the owner of the company has not been shown to be the best person (won't go into that here since it's not as tech related)

We used the mmap implementation (20 lines of code) which is also reflected in our attributions :)

https://github.com/wasmerio/wasmer/blob/master/ATTRIBUTIONS....

Perhaps I'm missing something. Were you referring to some other code I'm not aware of?

Yeah I think you're missing the part where you only added that file after being called out on twitter lol. The only reason I say "habits die hard" is because you consistently refuse to acknowledge that this ever happened.

If you can't watch the video, there's a blog post writeup of Lightbeam by the presenter here - http://troubles.md/posts/lightbeam/

> but for a streaming compiler... you just don't have liveness analysis

I understand that streaming compilation wants to start compiling before the entire module has been read, but is it really that important to start compiling functions before their entire definition has been read? Seems like a huge artificial hurdle.

functions are the part you compile, everything else is just runtime data

The question was why one would want to start compiling a function before all of its code is available.

Purely guessing, but: gigantic generated funcs are A Thing™, and pretty often I'd expect that you don't even need to know the full contents of a func to being executing it. Some structures would require it, sure, but not all. You could treat it as speculative execution up to the point where you can't, and then you just wait and continue-or-error when you know more.

Author here: you got here before me but this is precisely the right answer, although it’s more about not needing the full contents to start _compiling_, rather than executing necessarily. Supporting streaming execution parallel to compilation is absolutely possible, but a tougher challenge that isn’t necessarily for our purposes.

Why not... you could "just" during compilation split it into several functions and daisy-chain them if it came to that, by letting one calling the other with whatever state is needed to transfer. I'd bet it could even be part of a simple optimization pass. Single assign semantics help with eliminating useless scope and all that.

Reply on news.ycombinator.com