WASI 0.2.0 and Why It Matters
wasmcloud.com
wasmcloud.com
Because to me it seems only logical, to have POSIX that can work on the web.
The WasmCom keynote What is a component? (and why?) by Luke Wagner is also a great intro. https://youtu.be/tAACYA1Mwv4
Luke crushed it.
But they also mention components. Does that mean components are part of the web assembly standard now?
And am I correct in assuming that components are an option for projects where the functionality of WASI still isn't adequate?
When reading through all the things going into WASI Preview 2 (basically "everything and the kitchen sink", except for async/await(!) which goes into Preview 3) my first thought was: ok, this is what the second-system-effect looks like in practice.
> components interact only through the Canonical ABI. Specifically, unlike core modules, components may not export Wasm memory. This not only reinforces sandboxing, but enables interoperation between languages that make different assumptions about memory - for example, allowing a component that relies on Wasm GC (garbage collected) memory to collaborate with one that uses conventional linear memory.
Link: https://component-model.bytecodealliance.org/design/why-comp...
...for instance how do you share large amounts of data between components then, there must be some sort of cheap way to safely share portions of memory between components right? Because there are situations where multiple copy steps are simply out of question.
[0]: https://www.youtube.com/watch?v=tAACYA1Mwv4 [1]: https://github.com/WebAssembly/component-model/blob/main/des...
Yes, the component model is a standard developed under the umbrella of the W3C's WebAssembly Community Group.
That said, while it is relatively stable and hasn't changed much in the last year or so, the component model has not graduated through all the phases of the standardization process yet. It doesn't, for example, have a formal specification yet, although its canonical ABI does have a reference implementation in Python.
GC + async makes WASM a suitable runtime for languages like JavaScript and Dart so that we don't have to have JS + WASM runtimes where the JS doesn't integrate with WASI well, or WASI runtimes that don't support JS that well (compiling SpiderMonkey to WASM isn't great).
An additional benefit of using a fast JS runtime inside of WASM, even on a WASM engine that natively supports JS, is to use a Wasm Component as a security boundary around JS code. This could be used as an means of isolating supply-chain security issues, or allowing users to provide (untrusted) JS code that extends an existing system while having a clear and strong boundary on how untrusted JS code can affect the host JS code.
Or is my understanding of JavaFX incorrect, and it somehow turned applets into native binaries when you dragged them out of the browser?
Although now that there is actually a significant update to WASI, perhaps it's possible that they are just actually that slow? Or maybe both are true.
Probably I am just a "conspiracy nut". I mean I definitely am, but in this case it might just be paranoia.
At the end of 2023 we counted around 40 contributors who have been working on WASI specifications and implementations: https://github.com/WebAssembly/meetings/blob/main/wasi/2023/... . That is a great growth for our project from a few years ago when that issue was filed, but as you can see from what people are working on, its all much more foundational pieces than a graphics interface. Also, if you look at who is employing those contributors, its largely vendors who are interested in WASI in the context of serverless. That doesn't mean WASI is limited to only serverless, but that has been the focus from contributors so far.
By rolling out WASI on top of the WASM Component Model we have built a sound foundation for creating WASI proposals that support more problem domains, such as embedded systems (@mc_woods and his colleagues are helping with this), or graphics if someone is interested in putting in the work. Our guide to how to create proposals is found here: https://github.com/WebAssembly/WASI/blob/main/Contributing.m... .
"Web" indicates where it started / was standardized, not where it runs. WebAssembly runs everywhere, WebGPU does too.
And then gives code that reminds me of Enterprise JavaBeans from the early 2000's.
Thankfully, later on we get:
> "Let me show you an example of the ideal Wasm hello world application..."
And then have code that's clean, logical, and simple to read.
TBH, I almost gave up after seeing that first bit of code, but thankfully I kept reading. Having a standard interface will go a long way to creating a very nice future.
I've been trialing Preview 2 for the last couple of months to good results.
I do have one problem though and can't seem to find a good way forward. My use case is to use WASM for the data processing modules in a tile-based simulation game/engine. This involves passing around large (many MB) buffers. For any semblance of performance the data cannot be copied between modules and needs to be passed by reference.
To that end, I've implemented Resources. However, getting data out of a resource still requires copying (wasm-bindgen makes the return types from accessors owned Vec<>). I've resorted to just passing u32 memory offsets since all of the modules are sharing one Memory.
Would love some guidance on this as I've scoured the Internet and simply can't find the right solution, if there even is one.
As recommended elsewhere in the comments, there's a pretty damned fine talk my Luke Wagner that covers wasm components & the promise. He talks about cross-platform was 18m35s in: https://youtu.be/tAACYA1Mwv4#t=18m35s
What problem is it solving?
There are extensions to the core spec for embedding wasm. For example, there is "WebAssembly JavaScript Interface" for working with wasm inside JavaScript, which is what used by browsers today.
Spec we use in browser today is really just about how to interact with JS, so JS must provide everything that isn't in core: want to make an http request, then you need to call JavaScript (i.e. Fetch API) and so on.
People wanted to run WASM outside the browser because it's a neat abstraction - compile one of the many languages to WASM and run it "anywhere" (yay java).
Since outside the browser, we can't lean on JS for providing access to the outside world, plus that bridge has an overhead, so no bueno. WASI extends core spec with interface to outside system:
- files i/o
- network i/o
- etc
In addition, WASI has a built-in capability framework (a la Capsicum in FreeBSD which it drew inspiration from).
So WASI is a interface to a host system with security and isolation in mind.
I've been more of less following this for a decade ( since asm.js ), I still fail to see a practical use for this. And I mean a generalized use in real products and systems that stand the test of being an actual economically viable product, not cool demos which running Doom is probably the best one from a technical perspective.
For SO MUCH effort over a decade I feel it's more and more a nerd kingdom where it's full "of cool things" and much more work on creating and resolving problems that were solved years and decades ago and I still miss the point of all of this, I get the "big promise" but 10 years have passed and still nothing.
Also, I see a lot of Rust ( and I mean a lot ) attached to Wasm. Sorry but it's not going to happen, it just isn't. Rust is a systems programming language and Wasm has been pretty much another eternal tech demo. Real "mid" - as the gen-z says - use cases and products are needed and Rust is not going to cut it for a general product audience. This whole thing seems more like a "Run Rust in the browser" than a common runtime to run every language in the browser.
And since "running stuff on the browser" is kinda of old news, these new "wasm runtimes" ( which I get it and support ) in reality are basically a basic shitty proto-JVM.. again what's the point of all of this having spend 10 years?
The "devx" ( always a sucker for a new marketing term :) ) is HORRIBLE! Ever tried to compile a moderately complex ( and useful ) C program to wasm?
On a positive note, I wish success to Wasm/WASI because it's a cool idea and can open a lot of doors. If not for the actual reality and implementation of things, I'm optimistic about the general idea.
Sorry for my "ignorance" and if I hurt anyone's feelings.
As far as WASM going away: Sadly, I think browsers are just going to make up a larger and larger share of what computers do.
If I use Wasm in my native game then it’s easy for modders and others to create their own scripts and mods that can then be compiled to machine code by the game and safely executed on others’ computers. They don’t have to trust the modder because Wasm’s sandboxed design protects against a lot of bad behaviors.
It’s not so much that Wasm is totally unique here. Rather, to me it has many of the features and attributes I’m looking for in one single package/technology.
https://floooh.github.io/2023/12/31/vscode-wasm-wasi.html
(same with my home computer emulators, but those were written right from the start with the web as "just another platform" in mind: https://floooh.github.io/tiny8bit/)
Took me a couple of hours, mostly because of some pthread stuff I had to noop. And that's being fixed with wasi-threads
WASI-threads is not compatible with WASI-Preview2
https://floooh.github.io/2023/12/31/vscode-wasm-wasi.html
Also quite a few 'CPU nerds' found my 'visual6502 remixes' useful so far:
https://floooh.github.io/visual6502remix/
https://floooh.github.io/visualz80remix/
Development experience with Emscripten and WASI SDK is fine really (in the sense that these are "just another gcc-style cross-compilation toolchain"), and with those it's not much different than bringing a C code base to any other platform. It depends a lot on how portable the code is in the first place of course.
Not quite the same, but tangentially related since we're talking WASM+WASI and VSCode extensions - I have high hopes for the vscode-wasm[1] project. The VSCode extension host is a massive hog, and this seems like a big step in the right direction.
[0]: https://floooh.github.io/2023/11/11/emscripten-ide.html [0]: https://github.com/microsoft/vscode-wasm
Figma, Microsoft Flight Simulator, Disney+, Amazon Prime Video, Photoshop for Web, 1Password, and uBlock Origin all implement significant portions of their applications as WebAssembly modules.
Many of those examples are building their Wasm modules from Rust codebases.
From what I can see, Rust and Wasm have undeniably crossed the chasm into pragmatic, mainstream applications. The serverside Wasm ecosystem is still nascent, but that's what initiatives like WASI and the Component Model are designed to address. Yesterday's vote to launch WASI Preview 2 is a huge step towards building a stable, interoperable foundation for WebAssembly outside of web runtimes.
On the server side this is less clear. Typically we don't write system level languages server side - we use other languages, like Ruby, Python, JavaScipt etc. These interpreted languages can run in WASM, but only as language interpreter inside the WASM interpreter - so they work, but they are not efficient. The benefit to the cloud / infrastructure provider is often not-clear, not when it's possible to run these very same applications in existing infrastructure. On the server side the benefit to the application developer is also not clear. There are few providers who have infrastructure to support WASM applications directly. You'd really need to see the big 3 cloud providers look at supporting WASM directly, without hypervisors / vm / container technology under it, to realize any performance improvements and hence pass on some cost benefit to the developer ecosystem.
"running stuff in the browser"; isn't old news. There are a growing number of applications that are migrating toward web-based UI and for these WASM is really useful. WASI-Preview2's benefits are not going to be realized in a browser, it's more for the non-web world....
Rust-Focus; yes, you are correct the existing technology, with the exception of perhaps WAMR and CloudEdge runtimes are rust based. As WasmTime is the only standalone technology which currently supports preview2, and it's written in rust, then the associated examples and tooling are naturally rust focused.
The jco project (https://github.com/BytecodeAlliance/jco) provides an implementation of the Component Model and WASI Preview 2 for JavaScript systems. Right now, node.js support is complete, but support for Web embeddings is in progress and coming soon.
> These interpreted languages can run in WASM, but only as language interpreter inside the WASM interpreter - so they work, but they are not efficient.
The Bytecode Alliance has made big improvements to SpiderMonkey performance on WASM/WASI systems, and has work in progress to take advantage of SpiderMonkey's "native" codegen targeting WASM: https://cfallin.org/blog/2023/10/11/spidermonkey-pbl/. We targeted JS first for this work because it is the most popular language with our customers and users, but we expect that this will show the path to adding similar improvements to Ruby, Python, and other languages commonly thought of as "interpreted".
From my own perspective, which is browser based games, WASM opens up a lot of options. From using battle tested physics engines that are written in systems languages to the ability to access SIMD primitives. One of the underrated capabilities from that perspective to me is that floating point math is deterministic across browsers and platforms which is a bit of a holy grail for multiplayer games programming. Beyond that it promises to get faster and more efficient over time which is a good thing for performance heavy web apps.
Outside the browser MS Flight Sim uses WASM for the UI in part. WASI I know less about but doesn't seem totally crazy and in comparison to things like the JVM has a broader set of languages that can use it.
This is interesting and relevant to me. Do you have any suggested links where I can read more about this aspect of wasm?
HTML was built for the "internet", but works just as well for plain local files (eg: file:///one.html => file:///two.html).
JS was added to HTML, and much gnashing of teeth ensued.
JS (minified), HTML (generated), CSS (compiled), Flash, Java Applets, etc... move away from the "original" internet (Hyper-Text DOCUMENTS) into Web2.0 "Apps" and Web3.0 "Walled Gardens".
What if instead of shipping `my_application.html`, we could ship `my_application.exe`, and "break free" of needing the UI's being written in HTML, JS, CSS, etc.
What some of this WASM/WASI/etc... seems to be trending towards is building up enough of a foundation amongst interoperable _CLIENTS_ that have ZERO dependency on HTML, JS, CSS, and instead you get much closer to:
curl 'https://example.com/one.html'
=> curl 'wasm://example.com/one.app'
...to the extent that "the internet" has hit a complexity wall of what's possible (manageable, maintainable) with HTML+JS+CSS, the people tackling WASM seem to be saying:Forget trying to coerce a buggy browser into running my `for ...` loops (and rendering my graphics) correctly, and you can't realistically decompile or hand-edit the HTML+JS+CSS for 99.9% of the stuff your browser is rendering... how can we _skip_ the HTML,JS,CSS and get directly at "the VM" and run what we actually want.
Secondarily (but importantly, and interestingly): How can we make 'webapp.wasm.exe' support everything that a "native" app would need (eg: GL, GPU, Bluetooth, Local Files, USB devices, Camera, Sensors, etc...)
There's likely mega-trillions (no exaggeration!) of effort that's been poured into tooling for "the internet", what is the unifying effort/effect that would unlock that for local app development? How can I get `msword.exe` to run "on the internet" for free? How can I get `make clean ; make install ; make wasm` working for any internet connected client?
As a thought experiment, we're almost there! We could technically have `win95.img + bochs86vm.wasm + autorun.inf + msword.exe` wrapped in a "browser evaluator" that could conceivably run `msword.exe` transparently to the user, but still allow you to use `msword.exe` indistinguishably from the "original" native app (bridging the local filesystem, exposing virtual devices, etc).
See also https://www.destroyallsoftware.com/talks/the-birth-and-death... ... circa a decade(!) ago!
I looked into this and... holy crap! We are there. Not for modern programs quite yet, sure, but this is amazing. You can use Windows 2000 from your browser, running inside an x86 emulator for WASM.
Is is kinda heavy? Yeah. The hard disk footprint is substantial. But it works fine for writing some Python, compiling a few dependencies and bringing up a browser to test stuff. Even on a six-year-old laptop, it runs OK.
If the client application needs 1990's capabilities, "chuck it in a VM" will work today, and it will only be limited by protocol support, I/O access, and OS integration. There are substantial accessibility problems presented by VMs - seemingly basic things like clipboard support are disagreeable and require a "smart emulator" inserting itself to provide that feature.
Perhaps the right direction to take is to develop MAME to contain every accessibility feature. It already emulates all the old hardware.
What does this do that we couldn't do with Java Applets? So far the response is very few things and a lot less. On the few things that it does differently is: + it focus on multi language support. as opposed to the JVM which was focused in getting all of us in writing Java. Here they want to actually use different languages to write webapps. - Although Rust is the favorite (As is C# in the .net) + It has a clear mechanism to communicate with Javascript and therefore the web. + timing bandwidth is now big enough to make this apps seem practical. + it is done by a coallition of vedors (as opposed to java that was only Sun's baby).
Very minor advantages in my opinion... but well I also think that lip's s-expressions are better than json, xml and yaml. and the world has thought differently every single time.
Also applets were too slow to start with Sun's implementation of the day while wasm works reasonably well in all browsers from day one. That matters a lot for adoption.
JVM improved but applets missed their chance. You only get one.
WASI starts with nothing and then exposes a bit more. WASI2 exposes more than WASI1. Preview 3 will also add more. Eventually there will be sandbox escapes in WASM runtimes as well. There's nothing about WASM that makes sandbox escapes harder.
Bear in mind, applets exposed a large surface area, because you need that to write useful apps. Currently WASI looks useful for, maybe, some subset of CLI tools. Everything else is kicked to Chrome. It's not as ambitious.
https://en.wikipedia.org/wiki/Common_Language_Infrastructure
https://ecma-international.org/publications-and-standards/st...
> This Standard defines the Common Language Infrastructure (CLI) in which applications written in multiple high-level languages can be executed in different system environments without the need to rewrite those applications to take into consideration the unique characteristics of those environments.
If Microsoft was not aligned with developers in the early 2000's, that failure is on them.
Plenty of historical attempts to dive into, since 1958.
Now we have startups redoing Java and .NET application servers, with Kubernetes, WebAssembly, WASI, and YAML spaghetti, because that is so much better.
Edge devices running bytecode? That is so last century.
Edge devices running bytecode has its uses. Being an old idea doesn’t discredit it (neural networks were first described in the 70s)
At least WASI is a standard unlike flash or Java applets.
Maybe “open spec” or “open standard” is better.
The most exciting part about WASI for me, though, is that it's sticking to a capability-based interface and it's actually gaining a lot of traction. There are few examples of capability systems getting as much attention as this.
Is the idea mainly that permissions are more fine-grained? It probably makes porting existing applications tricky, though, right?
It is like microservices bandwagon nowadays, apparently distributed computing "Network is the Computer", computing agents, distributed objects, and service oriented architectures, also failed by the wayside.
Regardless I think WASM and WASI are huge step in right direction since most if not all relevant languages are going to support it.
If a new attempt can try the same idea, learn some lessons, and get more traction then I think that's a good thing.
Should we insult you with a huge list of rigorous enough things that were not designed in a paper? Or perhaps a list of unrigorous papers?
I hope you are not using a computer with a common OS and the tools it comes with because you might be in for some sadness and sorrow.
Or maybe your snark is unwarranted and you should take a step back on this.
I think nobody is claiming WASI is a brand new idea that is built from scratch without taking inspiration from anything old. On the contrary, it is immediately obvious to anyone with the most basic culture on the topic reading about WASI that it takes inspiration from previous work. And it's fine.
> Microsoft and its partners hold patents for CLI. Ecma and ISO/IEC require that all patents essential to implementation be made available under "reasonable and non-discriminatory (RAND) terms." It is common for RAND licensing to require some royalty payment, which could be a cause for concern with Mono. As of January 2013, neither Microsoft nor its partners have identified any patents essential to CLI implementations subject to RAND terms.
With such FUD, I guess no wonder CLI didn't have wider adoption.
The open source implementation of CLI was also left to the community, the main one being proprietary. If Microsoft & co wanted an actual usable standard, they could have done a better job.
Now, you are extending the scope to "all bytecode formats ever created" but that's not what I replied to and it makes your point a moving target. But fine, let's extend the scope, though I don't have a particular opinion on WASI, I haven't looked into it much.
How do you feel about the existence of multiple programming languages? CPU instruction sets? serialization formats?
Maybe WASI was specifically designed with the use case at hand and a new design was the better option. Saying previous work exists is not nearly enough. Most things are like this. They build on top of existing stuff taking inspiration from prior work.
I know nothing much about WASI. Convince me that an existing bytecode would have been better. Describe specific and detailed flaws.
Until then I'll just consider you just hate it for some unknown reason, so much you want others to join you.
Whatever you consider me to be, it is your opinion, keep it for yourself, share it with the world, whatever.
I have no particular opinion about you, in particular nothing against you.
But then Emscripten came along and showed how it's done properly, and everything derived from that (asm.js, WASM, WASI) was just evolution at work.
Last time I checked Emscripten was using Franken-clang.