Hacker News Reader

Comment by jeroenhd | Hacker News Reader

What I'm missing in these articles is a performance comparison. All WASMed tools I've tried were really cool proofs of concept, but the performance was always lacking at the very least.

I see several languages moving towards more and more WASM but on a technical level I don't see the benefit of WASM over something like Firecracker. Docker and other sandboxes have to deal with shared kernels and all the risks associated with that, but leveraging virtual machines instead solves that issue. There are already proof of concept implementations to replace Docker with VMs as a virtualisation layer, so I wonder if it wouldn't be better to invest time in getting those wrappers completely up and running rather than coming up with essentially "Java but we also emulate the OS".

Until WASM advocates start including benchmarks in their blogs, I'll keep watching this stuff from a distance.

View on HN

> Fast - it can offer native-like speed via the JIT/AOT capabilities of most runtimes. No cold starts, unlike booting a VM or starting a container.

What do you mean? This bullet point had a rocket emoji! Surely you don't actually want evidence to support a rocket emoji?!?

I don’t know about the rest of you but I’m pretty sure rockets are a couple of magnitudes faster then blue whales. Q.E.D. #

... rocket-propelled blue whales? (I mean, if we have sharks with lasers...)

Yeah, when I read this sentence, my thought was: how much slower does "fast" mean? :)

No cold-starts means no overhead of starting a process to answer a request like most container-based serverless environments (without having to keep pre-warmed instances which kind of defeats the purpose) A couple references regarding cold-starts and performance in serverless environments.

https://www.fastly.com/blog/lucet-performance-and-lifecycle https://arxiv.org/abs/2010.07115

Are we so what's-old-is-new-again as to be re-inventing fast-cgi at this point? And why are we pretending this has anything to do with WASM instead of just how your API/service is designed?

That’s basically the history of computing! There are some intrinsic advantages to using Wasm vs VMs or containers for certain scenarios, like serverless. That’s very similar to what’s going on with cloudflare workers and V8 isolates. It’s not one size fits all by any means, but it is certainly really good for many scenarios where containers are not

FastCGI reuses the same process for multiple requests. As I understand it wasmtime now supports very fast startup so you can use a new instance per request (avoiding the risk of inter-request bugs) with very low overhead (5 microseconds on their benchmark https://bytecodealliance.org/articles/wasmtime-10-performanc....)

With Firecracker I believe snapshot restore time is around 2-3ms. In my tests wasmtime ran about 50% the speed of native so depending on your workload it might still be faster for short running jobs where the startup time dominates. (Wasmer was maybe 80-90% of native speed but I don't know their startup times.)

> As I understand it wasmtime now supports very fast startup

WASM startup isn't going to be any faster than native code startup. It's going to be strictly worse if anything thanks to the JIT, although you can AOT that to native and then just restore parity with native code.

Which just gets back to the speed of your startup depends on what your startup does.

From what I understand, wasmtime's fast startup is conceptually similar to forking a process per instantiation, but much faster since it uses lazy initialization and has fewer operating system resources to setup.

Some of those techniques can be applied to native code too, see "On-demand-fork: A Microsecond Fork for Memory-Intensive and Latency-Sensitive Applications" https://www.cs.purdue.edu/homes/pfonseca/papers/eurosys21-od...

But I think wasmtime can always be faster to instantiate since the guarantees provided by the runtime allow it to safely reset and reuse instantiations:

"We implemented an “instance allocator” in Wasmtime that makes use of this copy-on-write (CoW) technique for very fast instantiations. It also uses a Linux syscall known as madvise to quickly “reset” the page mappings back to the original read-only heap image, so we can reuse the same mappings over and over when the same Wasm program is re-instantiated many times. (One might imagine this would be the case in a server serving many requests, for example!)"

https://bytecodealliance.org/articles/wasmtime-10-performanc...

> you can use a new instance per request

I doubt this will ever be as fast

What approach would be faster that provides similar isolation? It certainly seems a lot faster than fork and seemingly faster than the on-demand-fork I mentioned elsewhere.

It's native-like speed for some programs.

But it also depends a bit on the application.

Some applications can benefit a a lot from CPU specific instructions combinations which are not available to wasm (with available I mean implicitly, i.e. your wasm code gets compiled to them).

Luckily for a lot of use-cases this doesn't matter much(1) and some degree of SIMD support is often(2) available.

(1): Without micro-optimizations which most times aren't done as due to their maintenance/development cost.

(2): I'm not quite up to date. I think 128bit SIMD is available in most (all?) relevant WASI runtimes and at least some browsers.

@nine_k shared this https://programming-language-benchmarks.vercel.app/wasm-vs-r... in the comment tree. The results are pretty bad. You could lose 2x or more in cpu perf. There're cases where wasm is pretty close to native though.

I wouldn't call 50% loss "pretty bad". I mean sure, it's not great, but if you were to go from Rust to C# or Java you would most likely see a similar loss.

That would depend on the type of code you write. Heavily allocating code can be very fast in case of the JVM, and you can’t always avoid dynamic allocations/arenas are not always a solution.

yeah I have known about this for a while, but no one I've spoken to personally believes me. to them WASM is pure win and there are no downsides.

when I mention performance, they kinda waffle a bit, saying "CPU is cheap" or something similar, and they start to show a hint of understanding when I say that cloud resources are billed by unit of CPU time, and by amount of RAM used. then I say that our mutual employer invokes lambdas hundreds of trillions of times per year and I think they briefly understand before being caught up in "new stuff is awesome" technology fetishism again.

it's exhausting.

everyone should live overseas for a couple years because it changes how you view the world... everyone should be a game developer for a couple years as well, because you will quickly notice just how unbelievably slow modern software is. more people need to see that.

security is important! portability is important! other things are important, always, and when you gain a sense of just how slow software is today in comparison to how unbelievably fast modern hardware is, it becomes very hard for me to think positively of anything that lowers performance further for almost any reason.

The biggest missing thing in my mind is threading support. Great performance isn’t very useful if it only runs on one core.

> This allows for legacy applications to be ported to a browser and directly communicate with the JS code that runs in client-side Web applications.

Knowing nothing about WebAssembly, I would guess it's because JS runs on a single thread.

This is only true on the browser. Server-side JS has threads: https://nodejs.org/api/worker_threads.html

On the browser you have Web Workers and Service Workers, both run on separate threads :).

https://web.dev/workers-overview/#:~:text=Web%20workers%20an....

https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers...

https://developer.mozilla.org/en-US/docs/Web/API/Service_Wor...

Those are really more like separate processes. There's no ability to do a shared heap in browser JS, meaning it functionally doesn't have threads.

Whether or not workers are actually implemented as threads or processes in the runtime is irrelevant. As far as the JS code itself is concerned & what you can do with it, browser JS is lacking multi-threading. There's just no way to do a shared heap, and that is the biggest defining difference between a process and a thread.

The web has SharedArrayBuffer. It’s just difficult to work with.

Creating a shared memory allocation between 2 processes doesn't convert them to threads. The heaps are still distinct.

The JavaScript heaps are distinct, that's true, but there is a single shared wasm heap which is used from multiple threads. That is enough to implement the pthreads API.

Applications like Photoshop and Google Earth use ptheads on the Web so their compiled C++ is multithreaded, very similar to how it would run natively, and with similar responsiveness and throughput speedups. Though there are some limitations too, see

https://emscripten.org/docs/porting/pthreads.html

The entire WASM heap is a single ArrayBuffer (or SharedArrayBuffer) object.

Practically speaking, you just want shared memory in your threads. What would a shared heap offer that shared memory can't?

That's not really threading by the definition normally used in other languages. You can't allocate JS objects or structures and read/write them from multiple threads at once. JS is an inherently thread unsafe language and likely always will be.

Threading support is already implemented in some browsers and well on its way to standardization https://webassembly.org/roadmap/

Should address that concern. However there is another way of looking at performance and is in the context of serverless where typically single threaded performance is inportant, as well as cold start time etc and that’s why Wasm is popular in that scenario

“Threading support” in this sense is a bit of a misnomer, it’s really support for thread-safe memory constructs. Creating threads is left up to the runtime. On the web, this is done with WebWorkers, but on the server side I don’t think there is yet a standard way to do it supported by major runtimes.

No shared memory options?

There is a SharedMemoryBuffer, but it’s a web platform thing, not available in out-of-browser runtimes like wasmtime or wasmer or wasmedge (which Docker uses).

> Creating threads is left up to the runtime. On the web, this is done with WebWorkers

WebWorkers don't give you multi-threading behaviors (heaps/address spaces are not shared). WebWorkers would be how you launch a new process, but there's still otherwise no way to make a thread (nor even a fork() equivalent for that matter).

I think you could share an address space by using the same SharedArrayBuffer to back the linear memory of both?

I could be wrong here, I haven’t done it, but I thought this was the reason for supporting atomics in the first place.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

That only shares one allocation (like shared memory does in regular multi-process scenarios), but you still can't share the address space or even any object heaps at all. Like it's not possible to allocate javascript objects out of a SharedArrayBuffer such that you could pretend you had a shared address space by sticking everything in that.

As in, SharedArrayBuffer is equivalent to shm_open. Which means it's not even that good as a shared memory construct as it's missing all the protection enforcement of memfd (or Android's ashmem)

WASM stores everything in an array buffer, it doesn't use JavaScript objects because it's not JavaScript (though there are starting to be features that allow it to interoperate with JS objects). If it didn't store everything in a big memory array then it wouldn't really work because C assumes that

And no, WASM doesn't support memory protection

But shared array buffer is the underlying primitive generally for webassembly accessible memory.

Support on the server side is planned, still waiting for standardization https://github.com/bytecodealliance/wasmtime/issues/888

The biggest missing thing (for production) is observability.

Look at old-good JVM. It has tons of tools to analyze and understand behavior of your production system. You could have thread dumps (stack traces of all existing threads) at any moment with negligible performance impact, you could dump heap and analyze it off-site, you could have tons of metrics, about each dark corner of mutexes, GC process, about JIT, including, if you need it, generated native code!

Many of these thing you could get on production, not in sand-box.

If you system behaves strangely, live-locks, consume more memory than you think it should, tharsh GC, you name it, you have all tools to understand what is wrong, find bugs or mis-configurations, etc.

With all these new-and-shiny WASM and not-so-shiny JS VMs you mostly in the dark now. Service become unresponsive? latency goes to the roof? Only thing you could do - restart.

It is not property of WASM per se, but this infrastructure is too immature now, comparing to 25+ year old technology.

This a 1000 times. People like to hate on Java but when there are problems to diagnose on production systems it is second to none.

But from my experience most people don't know these tools even exist so the only thing they do is restarting and guessing where the problem might be if it persists.

Flight recorder is a godsend and I've not seen it's equal in any other language/ecosystem.

Any JVM anywhere can answer the question "why am I running slow" with a quick run of flight recorder. Memory, CPU, socket time, GC impact, TLB, thread dumps, etc. It's all there in one file that imposes something like a 1->2% performance impact if you run it constantly.

It's just so good.

Completely agree. Observability and debugging are some areas where the ecosystem is quite immature or inexistent. My take is that wasm is more or less where the web was in 97-98 Lots of excitement and possibilities but also lots of technical challenges and experimentation

"nice" threading support is not there

but you can have threaded wasm code in any evergreen browser since a more then a year as far as I'm aware

Basically the trick is that you use multiple web-workers with the same WAS program and the same shared buffer. Then you also add some JS glue code to coordinate which thread is the main thread and which threads you use as thread pool (e.g. in rust/wasm with rayon you can set it up as worker pool).

Now there are some drawbacks (last time when I used it, might have gotten better):

- threads are started/managed from outside (so don't expect any kind of "spawn" function to work, generally spawning new threads is non-trivial and so is (properly) cleaning up old threads, through if you need a fixed worker pool it's all fine)

- there where some limitations wrt. threading/synchronization which made certain usages of concurrency rather slow (through many where fine)

- no "synchronized" operations mustn't be called from WASM code called by the main JS thread. This means in most situations you need to pass data to web workers and then to WASM (instead of e.g. passing it to WASM and then using in wasm a mpmc-channel to pass it to the worker pool). There are some optimizations around passing pointers as numbers to/from the web-workers but it's limited and not nice. Or at least wasn't ~a year ago.

- bugs in Safari leading to strange crashed for code running in all other browsers nicely under unclear and non-debuggable circumstances (probably fixed, I hope)

Anyway all in all using rust->wasm with rayon and a thread pool was already surprisingly viable ~1 year ago.

Not exactly what you've asked for, but I did a "dummy" benchmark of Rust vs WASM on my blog[0]. WASM is impressive technology indeed, and it will be interesting to see whether it will get into the mainstream of software engineering.

[0] https://www.yieldcode.blog/post/native-rust-wasm

> but the performance was always lacking at the very least.

This is an easy engineering problem which will be solved when there's enough motivation and engineers working on it.

Increasing adoption is more of a business problem though, and it is unclear if performance is the bottleneck here.

If you have a wasm product that has to be as fast as native code, the solution is to find compiler engineers (or a company that specializes in this) who will solve this for your situation.

Bro we are hearing about this sufficient smarts compiler for some decades now.

You are right when it comes to general purpose compilers.

You can do a lot more if you just want to speed up your codebase.

All the big tech companies employ multiple hundred compiler engineers each for this purpose.

I started skimming and skipping because that's the only thing I was really interested in.

Reply on news.ycombinator.com