Hacker News Reader

Comment by kentonv | Hacker News Reader

It's a very minor incompatibility:

Traditionally, all WebAssembly modules are essentially eval()ed. You need JavaScript to download the module into an ArrayBuffer or the like, then pass that to the WebAssembly API to compile it.

However, in Cloudflare Workers, we didn't want you to have to fetch WebAssembly remotely at startup. Instead, you upload your WASM module to the Cloudflare configuration UI/API together with your JavaScript code. At startup, the WASM is compiled and the resulting `WebAssembly.Module` appears as a global variable in your script, which you can then instantiate.

Emscripten normally automatically generates JavaScript for you to load your WASM. But Emscripten's generated script doesn't understand this delivery model where the module shows up as a global variable. It should be trivial to add support, but it would be awkward for us to try to submit a patch upstream without the functionality being public yet.

View on HN

Actually, emscripten has a built-in flag (SINGLE_FILE) to embed the wasm code as base64 (which I'm the author of, incidentally).

May I recommend also supporting hex encoding the WASM? I think you'll find that while your file size is larger, it's actually significantly smaller once gzipped.

Edit: Just tested this on https://public.tableau.com/vizql/v_public-release1809140800/...

  runtimeweb.wasm       2,484,043
  runtimewebwasm.b64    3,355,640 
  runtimwebwasm.hex     4,968,086
  runtimewebwasm.b64.gz   974,065
  runtimewebwasm.hex.gz   701,052
  runtimewebwasm.b64.br   718,918
  runtimewebwasm.hex.br   466,221

So with gzip -9, hex encoding is 72% of the size, with brotli (defaults) size is 65% of the size.

Hmm, thanks for the tip, and thanks for testing that out so I don't have to! It hadn't occurred to me that that would be the case, but it makes some sense after reading through https://stackoverflow.com/q/38124361/459881. I'll go ahead and open an issue with emscripten about this.

Yep, it's one of those counter intuitive things. Another benefit from using hex is the decoder is a lot simpler and easier for the JIT to vectorize.

That actually doesn't quite solve the problem as we disallow dynamic code evaluation. This is for a couple reasons:

* Security: In case of an attack we need to be able to call up the code to do forensics, which is hard if it was downloaded dynamically at runtime.

* Optimizations: We want to keep the ability to pre-compile code e.g. into V8 code cache format before distributing it to the edge.

Ah, I thought that might've been part of it since you brought up the comparison to eval; just wanted to clarify that there are other options for avoiding reliance on external subresources.

which is hard if it was downloaded dynamically at runtime

Not sure if you mean something other than what this sounds like (I haven't used CF Workers), but just to clarify further, the option I mentioned causes emscripten to emit one single JS file with no external code downloaded at run time.

(In any case, that performance optimization alone certainly sounds like a good reason for the separation.)

Understood. While the patch is being mainlined, it would benefit emscripten users targeting CloudFlare to use whatever you have working with emscripten's build workflow. Please consider releasing this repo for the interim

We haven't actually tried making any changes to Emscripten yet. We've only used lower-level tools to build things more manually. There is a working example of this here:

https://github.com/cloudflare/cloudflare-workers-wasm-demo

What about Module.instantiateWasm()? That should let you provide a pre-compiled or even pre-instantiated module.

Reply on news.ycombinator.com