Hacker News Reader

Comment by pcwalton | Hacker News Reader

Having been on one side of the perpetual (and tiresome) PNaCl-versus-asm.js debate, I'm thrilled to see a resolution. I really think this is a strategy that combines the best of both worlds. The crucial aspect is that this is polyfillable via JIT compilation to asm.js, so it's still just JavaScript, but it has plenty of room for extensibility to support threads, SIMD, and so forth.

View on HN

> The crucial aspect is that this is polyfillable via JIT compilation to asm.js

So is PNaCl, with pepper.js. The difference there is that PNaCl also provided a more full-featured API, which nonetheless was harder for other browsers to support. Personally, I would have been happy to see NaCl-minus-Pepper standardized, since the sandbox would have been much easier to support than the APIs; browsers could then choose what superset of APIs to expose to that sandbox.

By "a more full-featured API" do you mean that Pepper has more features than the Web APIs? If so, then it's not polyfillable (and the solution is to add more features to the Web APIs). Or if you mean that PNaCl has more features than asm.js, then the solution is to add those features to JavaScript.

I'm glad that asm.js is the starting baseline for this work, because its exact semantics are standardized via ECMA-262. All that has to be done to define its behavior precisely is to create a new syntax for it. LLVM bitcode, by contrast, has a lot of undefined behavior. You could try to spec a subset of it, but why go to that trouble when TC-39 has already done the work for you?

Yes, Pepper has more features than existing web APIs, though in some areas web APIs are finally catching up.

> Or if you mean that PNaCl has more features than asm.js, then the solution is to add those features to JavaScript.

Nope. A polyfill is a temporary solution, and will not be a permanent design constraint. I'm very glad to see that the constraints of JavaScript will no longer be a limiting factor on the web. I look forward to the day (already anticipated in the linked post) when a JavaScript polyfill is no longer a design constraint.

What features in particular are you missing from the Web platform?

I see no good reason to add features to Web Assembly and not to JS. A Web Assembly engine is virtually always going to also be a JavaScript engine, so the implementation effort needed to keep them at parity will be minimal. And the demand from Web authors to add all the new Web features to JS will remain high. The likelihood of Web developers suddenly fleeing JS is comparable to the likelihood of WordPress migrating away from PHP. It's just not going to happen.

> What features in particular are you missing from the Web platform?

In terms of language: Myriad features from C, Python, Rust, and other languages. Static typing. Integers. Tagged unions. Real data structures. Efficient native code (asm.js does not suffice), which would allow implementing those.

In terms of libraries, a few off the top of my head: OpenGLES (no, not WebGL), real file APIs (both provider and consumer, and including memory-mapped files), raw TCP and UDP sockets (not WebSocket), a better and more efficient low-level rendering API...

> A Web Assembly engine is virtually always going to also be a JavaScript engine

Definitely not; browsers will include both for the foreseeable future, but a WebAssembly engine/JIT will be substantially different from a JavaScript engine/JIT, and compilation for WebAssembly won't go by way of JavaScript in browsers with native support. The article itself says the polyfill will be a temporary measure.

> And the demand from Web authors to add all the new Web features to JS will remain high.

> The likelihood of Web developers suddenly fleeing JS is comparable to the likelihood of WordPress migrating away from PHP.

There are web developers who actively like JavaScript, and there are web developers eagerly desiring another language; the latter motivates things like CoffeeScript, TypeScript, Dart, Emscripten, asm.js, Native Client, and many more, none of which would exist if every web developer was completely thrilled to write nothing but JavaScript.

And on the flip side, people who love JavaScript have created environments like node.js so they can use it outside the browser.

Everyone who loves JavaScript will keep using it, and it certainly won't go away. And JavaScript will be able to call almost all new web APIs. But not everyone will keep using JavaScript.

WordPress won't migrate away from PHP, but plenty of sites choose not to use WordPress.

Those language features aren't relevant to Pepper vs. the Web APIs: JavaScript supports integers, tagged unions, and so forth, if you compile a language that supports them down into JS. As for the library features, aside from a few I don't understand/agree with (how is WebGL not equivalent to OpenGL ES, and how is WebGL/Canvas not a low-level rendering API?), a bunch of those are forbidden for security reasons (like raw file access and raw TCP and UDP sockets), so you wouldn't want pages to just have access to them in the manner that a PNaCl-using Chrome extension does anyway.

> a bunch of those are forbidden for security reasons (like raw file access and raw TCP and UDP sockets), so you wouldn't want pages to just have access to them in the manner that a PNaCl-using Chrome extension does anyway.

Sure I do, with explicit permission granted. For instance, consider a web-based VNC client, without needing a server-side WebSocket proxy.

And there are far more where those came from.

Also, the advantage of making low-level web APIs is that any new high-level APIs built on top of them run inside the sandbox, so their implementation is not part of the attack surface.

Considering that people already are installing apps on their smartphones and nod away all the checkboxes

   [x] share all my contacts
   [x] rummage through my images
   [x] tap into microphone and camera at any time
   [x] set my house on fire

such a list of allow/deny simply needs to come to browsers too.

Ideally they should be designed in a way that those feature unlocks get delayed randomly, so that they always "break" any code relying on those APIs by default. I.e. stuff should still work if the user doesn't give you the key to the city.

We had this, it was called "Java applets" (also, DirectX), and it sucked.

Developing on the Web platform downright sucks compared to pure native environments. There are so many things wrong or missing, it's hard to know where to start.

The lack of any kind of standard library comes to mind. The ability to use something other than Javascript, without having it compile to Javascript, would be fabulous. Being able to use something, anything other than a hodge-podge of HTML/CSS and JS for making a UI would be wonderful.

Sooner or later someone is going to invent a browser that can browse something other than HTML and HTML, CSS and JS will become legacy.

it's not polyfillable

I'd be interested to know where/when this "polyfillable" term came about. It's not readily Google-able yet.

https://remysharp.com/2010/10/08/what-is-a-polyfill

If you dig through my comment history, you might be able to find a comment I wrote on my first exposure to this term maybe ~2 years ago. I was utterly befuddled because I thought it was a graphics rendering algorithm (setting pixels in a framebuffer corresponding to the interior of a polygon specified as an edge list), and was horribly confused about why an article discussing some web technology would suddenly transition into a geometric problem utterly disconnected from the original topic.

It's a pretty awful term...

It sure beats "shiv" though. Glad that one has started fading.

shim makes much more sense than shiv

Spackle would have been a fun term.

> I would have been happy to see NaCl-minus-Pepper standardized[...]

If there was no standard API, then sites would have a different API they had to use for each browser. Then we're back to the `UserAgent` days.

Unfortunately, I think it must be an all or nothing ordeal :/

> [...]browsers could then choose what superset of APIs to expose to that sandbox.

Pepper can already do that. All Pepper APIs are retrieved by calling a function provided to the module at start with a single string argument (ie `"PPB_Audio;1.1"` or `"PPB_Audio;1.0"` or `"PPB_FileSystem;1.0"`). It's impossible to use PPAPI without first getting the specific interface structures from that function.

> If there was no standard API, then sites would have a different API they had to use for each browser. Then we're back to the `UserAgent` days.

You would need an API to send and receive messages between NaCl and JavaScript, but otherwise you could get away with just a sandbox that runs native code. Everything else could be incremental.

pepper.js is ahead-of-time offline compilation, which means it can't adapt to individual target runtimes.

So is WebAssembly: you'd compile your code to a .wasm file "offline". In both cases, you compile your code to a bytecode format, which the browser then compiles (directly or via JS polyfill) to native code.

A key part of WebAssembly's strategy is a polyfill that runs on the client and generates appropriate JS (asm.js or otherwise).

Right, there will be a wasm.js polyfill just like pepper.js, for browsers without native support. And once browsers have native support, that polyfill can be dropped.

The difference between PNaCl and WebAssembly is simply that WebAssembly doesn't come with a pile of API expectations established by a single browser, and that it has the blessing of multiple browser vendors. Ignoring the API, the latter could have worked with PNaCl just as well.

Any idea how this will end up working in practice? If it will actually require linking to an external .wasm file, that'll break the way we're currently using asm.js.

I'd like to see something that works more or less to the effect of:

    const wasmCode = new Uint8Array(...);
    
    if (typeof executeWasm !== 'undefined') {
        executeWasm(wasmCode);
    }
    else {
        compileToAsmjs(wasmCode);
    }

The current plan (https://github.com/WebAssembly/design/blob/master/Web.md#imp...) is to integrate WebAssembly into the ES6 module system (so you could have a JS module import a WebAssembly module and vice versa). With this design, the WebAssembly polyfill would hook into the ES6 module loader (if implemented natively) or the ES6 module loader polyfill.

It should also be possible to have a <script type="application/wasm" src="something.wasm" />, and then additionally source a JavaScript polyfill that checks if wasm is supported and translates to JavaScript if not. That would remove the need to have JavaScript shims.

Hmm, neither one of these options (ES6 import or <script> type wasm) can quite fit in with the way we're using asm.js (including it inline with other JavaScript inside a Worker[1]).

If there were a way to flexibly invoke wasm in any scenario where asm.js currently works, I think it would be a lot friendlier and truer to the "it's just JavaScript" goal.

Furthermore, tying this to ES6 seems unnecessary to me. How does this affect those of us using Babel or TypeScript to output ES3 code? The purposes of wasm and ES6 just strike me as completely orthogonal.

---

1: There are good reasons we have to do this, but they're beyond the scope of this thread.

> Hmm, neither one of these options (ES6 import or <script> type wasm) can quite fit in with the way we're using asm.js (including it inline with other JavaScript inside a Worker[1]).

Could you import a module, and then call the resulting functions from within your Worker?

> If there were a way to flexibly invoke wasm in any scenario where asm.js currently works, I think it would be a lot friendlier and truer to the "it's just JavaScript" goal.

"it's just JavaScript" is a goal of asm.js; I don't see anything about WebAssembly that makes it a stated goal of wasm. "It works anywhere JavaScript works, and can call and be called by JavaScript" would be a more sensible goal.

> 1: There are good reasons we have to do this, but they're beyond the scope of this thread.

I'd be interested to hear them.

Not sure if the security model allows it, but maybe you could write the wasm into a blob, create an URI from the blob[1] and then use that as <script> src.

Or maybe data uris.

> How does this affect those of us using Babel or TypeScript to output ES3 code?

Well, if you're already transpiling the whole load of ES6 features then it's just one more shim to consider, no?

[1] https://developer.mozilla.org/en-US/docs/Web/API/URL/createO...

Any idea if wasm / ES6 modules / module loader shims work with object URLs and/or data URIs cross-browser by design?

Doing something similar for Workers, one of the problems we've had is that IE and I think Safari block that, so to make it work we have to fall back on a permanently cached shim that puts an eval statement inside self.onmessage. (Obviously no equivalent to eval exists for wasm, or I wouldn't have a problem here in the first place.)

I wouldn't say this is "tied" to ES6, but rather intends to integrate nicely. If a developer has no interest in being called by or calling JS, they should be able to ignore the ES6 module aspect. For workers, it should (eventually, probably not in the MVP v.1) be possible to pass a URL (which, with Blob + Object URL needn't be a remote fetch and can be explicitly cached (Cache API or IndexedDB) or dynamically generated (Blob constructor)) to a worker constructor.

>Could you import a module, and then call the resulting functions from within your Worker? ... I'd be interested to hear them.

I have more detail here[1], but what it comes down to is that our application's security model is dependent on the ability to pack all of its code into one file.

I'm admittedly not too familiar with the ES6 module system (we're using TypeScript modules), but it looks like importing necessarily requires pulling in an external file. Workers are problematic in a similar way, but simple enough to work around using a few different methods (that I can't think of how to apply to ES6 modules).

> For workers, it should ... to a worker constructor.

In our case, the asm.js modules aren't being used as standalone workers, but rather pulled in to a worker that does a bunch of other stuff, so that probably couldn't be applied here exactly.

If that blob URL setup worked with ES6 import, though, that could work. (It might make things messier for generated TypeScript code though, not sure.)

> I wouldn't say this is "tied" to ES6, but rather intends to integrate nicely.

When I say tied, I mean in the sense that ES6 looks to be the only method of arbitrarily executing wasm code.

To use wasm, I'd need the ability to drop it in the middle of a block of JS.

---

1: https://docs.google.com/document/d/1j5KnpVyDdIXVwEDCQpHGxbs0...

Almost nobody is using asm.js right now, and backward compatibility with asm.js should not be a top consideration for WebAssembly which (with agreement from multiple vendors) will achieve something much more important than asm.js backward compatibility.

My point is less about anything related to asm.js per se, and more that there isn't a good reason for wasm not to support this particular use case.

With this module loader, can you generate your own code to load at runtime reasonably easily and efficiently? Say you're compiling from user input to asm.js within the browser (as I did the week asm.js was announced) -- can you change it to compile to wasm?

(That link pointed to another link on module loading, a long page I didn't find an answer on right away. Admittedly I'm only curious for now.)

Added: https://github.com/lukehoban/es6features#module-loaders shows module loading of runtime-generated JS code, so the answer is probably yes, hurray.

That'd be perfect; thanks! I'll have to play around with that once wasm is in a testable state.

As far as compiling within the browser, it sounds like compiling the wasm will be the same process as compiling to asm.js – except the final bytecode format won't be directly executable as JS. If that's the case, then I'd imagine any asm.js compiler could have wasm support added the same way emscripten will.

I think in both of our use cases it all just depends on how easy it will be to execute the code from there. (In your case, the worst case scenario is that you have to post the generated code to a server and redownload it – ridiculously ugly, but not a blocker.)

In the Java applet days you'd actually have to do that -- round-trip your generated bytecode back through the server -- and I was like "Hello, you must be kidding". That's why I don't just assume they've made direct loading possible.

I haven't followed the module loader work super closely, but the overriding impression that I've gotten is that the ability to customize how code is loaded and from where is a primary goal.

What's wasm's optimization story? Is there an optimizing JIT running on wasm code, or is the code supposed to already be optimized by the AOT compiler (in which case, wasm is less appropriate as a target for languages with very dynamic dispatch, which should still compile to JavaScript)?

EDIT:

I see an optimizing JIT is planned for future versions: https://github.com/WebAssembly/design/blob/master/FutureFeat... while the MVP is primarily meant to serve as a target for C/C++.

There are two different levels of optimization involved here. Since WASM isn't the native assembly language of any processor, the browser's JIT will need to translate WASM to native machine code, including efficient translation of SIMD and other instructions. Separate from that, compilers for various languages will need to have WASM backends, including the WASM constructs that translate to efficient machine code on various target platforms.

Sure, but there's a big difference between simply JITting WASM to native code trivially or with minor optimization, and serious profile-guided/tracing optimization that's required for efficient execution of languages heavily based on dynamic dispatch. From what I gather from the design documents it seems that the goal is to start with straightforward JITting (aimed at languages like C), and later add a more sophisticated optimizing JIT.

Sounds like it to me. High-performance JavaScript JITs have to rediscover type information that doesn't exist in the source language. WebAssembly appears to have enough information to make that mostly unnecessary, so much of that optimization can move into the compiler generating WebAssembly.

> so much of that optimization can move into the compiler generating WebAssembly.

Yes, but JavaScript can't be compiled to efficient WASM code. Dynamic-dispatch languages need an optimizing JIT -- compiling once runtime behavior is known -- to produce good machine code. In any event, it seems like it's a planned feature for a some future release.

Well, you at least need something like an inline-cache instruction (like invokedynamic). You also need GC hooks so the stack layout, object layouts, and pointer liveness are known. I don't see any reason why it couldn't be done in the future.

(We may not be in disagreement here—I agree that JS can't be effectively compiled to Web Assembly as it stands.)

No argument :) I just asked a question and then answered it myself based on what I read.

The design documents clearly state that WASM is intended to be a compilation target for C-like languages, and in the future may also contain a GC and an optimizing JIT.

Languages requiring a GC and/or not amenable to AOT optimization should -- at least for the time being -- continue to target JS as a compilation target (or implement their own runtime in a C-like language and embed it in WASM).

If I'm reading this correctly, they are not planning to add a JIT but just the features required for implementing your own:

    WebAssembly should support:

    Producing a dynamic library and loading it into the current WebAssembly module.
    Define lighter-weight mechanisms, such as the ability to add a function to an existing module.
    Support explicitly patchable constructs within functions to allow for very fine-grained JIT-compilation. This includes:
        Code patching for polymorphic inline caching;
        Call patching to chain JIT-compiled functions together;
        Temporary halt-insertion within functions, to trap if a function start executing while a JIT-compiler's runtime is performing operations dangerous to that function.
    Provide JITs access to profile feedback for their JIT-compiled code.
    Code unloading capabilities, especially in the context of code garbage collection and defragmentation.

From https://github.com/WebAssembly/design/blob/master/FutureFeat...

Right, because wasm is for AOT (ahead of time) languages first. To support dynamic languages that don't map to JS well, the future extensions including JITting wasm will be needed.

Reply on news.ycombinator.com