syrusakbary

Fully agree with your take.

I have no idea what the people guiding WASI are motivated about, but for sure it doesn't seem aligned with the community motivations... which might be hurting the ecosystem in the long run.

For the exact reasons the parent comment mentioned, at Wasmer we aimed for full POSIX compatibility as part of WASIX (a superset of WASI 0.1): https://wasix.org/. WASIX has support for many syscalls not available in WASI: fork, exec, longjmp, setjmp, threads, sockets, and many more. It's insightful to know that many people, and specially those close to the Bytecode Alliance, neglected the contributions, and try to push down the project, and even tried to make us rename it: https://github.com/wasix-org/cargo-wasix/issues/4

I see [1], thanks for sharing. I'll need to dig a bit deeper in your implementation!

[1] https://github.com/wasmi-labs/wasmi/blob/master/crates/wasmi...

Great analysis on all the runtimes.

I love all the improvements that Wasmi has been doing lately. Being so close to the super-optimal interpreter Stitch (a new interpreter similar to Wasm3, but made in Rust) is quite impressive.

As a side note, I wish most of the runtimes in Rust stopped adopting the "linker" paradigm for imports, as is a completely unnecessary abstraction when setting up imports is a close-to-zero cost

I've been following Tantivy for a little while. It's impressive the grit that the founders have, and the performance that Tantivy has been able to achieve lately.

Mad props to all the team! I'm a firm believer they will succeed on their quest!

Author here. Faster than it has ever been in the Edge via WebAssembly :)

But you are completely right pointing out that there's still some room to improve, specially when compared to running PHP natively.

Right now there's some price to pay for the isolation and sandboxing, but we are working hard to reduce the gap to zero. Stay tuned for more updates on this front!

It's refreshing to see that we are not alone in our thoughts on how the community is not being stewarded towards its own interests. I applaud the author on how clear he made the argument.

For those that aim to continue working on top of WASIp1, WASIX (https://wasix.org) might be a great way to get your programs with sockets and threads fully running on Wasm.

Note: I work at Wasmer (https://wasmer.io), a WebAssembly runtime.

This is really awesome. Mad props for Chris for working on weval, and to Max for the great overview with the blogpost.

I found the video in Vimeo quite useful to understand how it works: https://vimeo.com/940568191

Thanks!

That should help us a lot to investigate further the difference in timing. I remember trying the py2wasm strategy about 4 years ago, and I got it running faster than native CPython, so there must be something we are missing!

Indeed, this can be way further optimized. For example, you can probably do a wasm-strip and wasm-opt passes that would leave the wasm file being ~5-10Mb. Still very big, but a bit more reasonable.

The good thing is that thanks to Nuitka you could actually do some tree shaking and only include the imported files and not everything in between (although this might break other behavior such as eval and so).

Hey, I'd love to reproduce the ~2x slowdown you commented from running the workload in Native CPython compared to Wasm CPython (in any runtime, browser or outside).

Any tip would be helpful so we can debug it. If your claims are accurate, we can easily get py2wasm even running faster than native CPython!

Note: we benchmarked in a M3 Max laptop, so maybe there's some difference there?

Thanks for the feedback! I'm Syrus, main author of the work on py2wasm.

We already opened a PR into Nuitka to bring the relevant changes upstream: https://github.com/Nuitka/Nuitka/pull/2814

We envision py2wasm being a thin layer on top of Nuitka, as also commented in the article (not a fork as it is right now, although forking got us into the proof of concept a bit faster!).

From what we gathered, we believe that there's usefulness on having py2wasm as a separate package, as py2wasm would also need to ship the precompiled Python distribution (3.11) for WASI (which will not be needed for the other Nuitka use cases), apart of also shipping other tools that are not directly relevant for Nuitka

It also even works in Wasmer Edge (template here: https://wasmer.io/templates/hono-starter )... I'm incredibly impressed with how much effort they have been putting on the framework, I bet they will win big time!

> Wasmer claims to cold start in 50ns

I believe this statement was based on Wasmer Edge product page, which was mainly measuring instantiation time if the module is already loaded in the Edge Node (which can be assumed true for most common programs such as WinterJS, static-web-server and more). As it seems the 50ns timing was not well understood, we updated our product page to reflect a more accurate timing thanks to your feedback.

We can get into sub-millisecond cold starts (loading a module from disk, instantiating it and serving the first request) for the best case scenario, a timing now properly reflected in the Wasmer Edge product page.

In any case, and to be clear, we can get into much faster cold starts than the full second than Cloudflare Workers currently offers for Python. Wasmer Edge should be at least 10x faster than Cloudflare Workers for Python cold starts, with a cold-start time of less than <100ms. You can also expect to see much faster cold start times in next releases :)

As a side note, Wasmer offers an Edge product that has none of the drawbacks commented when running Python in Cloudflare Workers, providing incredibly fast cold-start times:

https://wasmer.io/templates?language=python

Gaussian splatting is truly amazing for 3d reconstruction.

I can't wait to see once it's applied to the world of driverless vehicles and AI!

Hey Kenton, great to see you chiming in here as well!

> Additionally, V8 is just much better at running JavaScript than you can hope to achieve in a Wasm-based JS implementation. And JavaScript is the most popular web development language (even server-side).

I agree with this statement as of today. Stay tuned because very cool things are coming on Wasm land (Spidermonkey will soon support JITted workloads inside of Wasm, bringing the speed much closer to V8!)

> Note that the time to initialize an isolate isn't actually on the critical path to cold start, since we can pre-initialize isolates and have them ready to go before knowing what application they will run

That's a good point. Although, you are kind of optimizing now the critical path to cold start by actually knowing what the app is running (if is Python, restore it from a Snapshot). So even though if isolate initialization is not in the critical path, there are other things on the critical path that amounts for the extra second of latency in cold starts for Python, I would assume.

> Minimizing application size is really essential to making edge compute inexpensive

By leveraging on proper-defined dependencies, you just need to compile and load in memory the dependency module once (lets say Python) and have "infinite" capacity for initializing them. Basically, if you put Python out of the picture and consider it a dependency of an app, then you can suddenly scale apps as much as you want there!

For example: having 10 Python versions (running thousands of apps) will have a overhead of 5Mb (Python binary size in avg) * 10 versions (plus a custom memory for each initialization of the app, which is required in either strategy) ~= 50Mb, so the overhead of pinning a specific Python version should be truly minimal on the server (at least when fully leveraging on a Wasm runtime)

Awesome. Eager to see how the product evolves :)

> Your concerns about V8 would impact JavaScript Workers as well and do not match what we see in production

Interesting! I thought V8 snapshots were mainly used in the Pyodide context, as I could not find any other usage in WorkerD (other than promise tagging and jsg::MemoryTracker).

Are you using V8 snapshots as well for improving cold starts in JS applications?

> pyodide /package versions will be controlled by the compatibility date

That's exactly the issue that I'm mentioning. Ideally you should be able to pin any Python version that you want to use in your app: 2.7, 3.8 or 3.9 regardless of a Workerd compatibility date. Some packages might work in Python 3.11 but not in 3.12, for example.

Unfortunately, Python doesn't have the full transpiler architecture that JS ecosystem has, and thus "packaging" Python applications into different "compatibility" bundles will prove much more challenging (webpack factor).

> Could you expand on why you believe V8 will be a limiting factor?

Sure thing! I think we probably all agree that V8 is a fantastic runtime. However, the tradeoffs that make V8 great for a browser use case, makes the runtime more challenging for Edge environments (where servers can do more specialized workloads on trusted environments).

Namely, those are:

  * Cold starts: V8 Isolates are a bit heavy to initialize. On it's current form it can add up from ~2-5ms in startup just by initializing an Isolate
  * Snapshots can be quite heavy to save and restore
  * Not architected with the Edge use case in mind: there are many tricks that you can do if you skip the JS middleware and go all in into a Wasm runtime, that are hard to do with the current V8/Workerd architecture.

In any case, I would love to be proven wrong on the long term and I cheer for <100ms cold starts when running Python in Cloudflare Workers. Keep up the good work!

As far as I know uvloop is not supported in Pyodide, mainly because it requires compiling libuv into WebAssembly (which is possible but not trivial).

In any case, it shall be possible to run uvloop fully inside of WebAssembly. However, doing so will prove challenging using their current architecture

This is awesome, I'm happy that Cloudflare is adding more attention into running Python via WebAssembly at the Edge.

I'll try to summarize on how they got it running and what are the drawbacks that they have from their current approach (note: I have deep context on running Python with WebAssembly at the Edge as part of my work in Wasmer).

Cloudflare Workers are enabling Python at the Edge by using Pyodide [1] (Python compiled to WebAssembly via Emscripten). They bundled Pyodide into Workerd [2], and then use V8 snapshots [3] to try to accelerate startup times.

On their best case, cold starts of Python in Cloudflare Workers are about 1 second.

While this release is great as it allows them to measure the interest of running Python at the Edge, it has some drawbacks. So, what are those?

  * Being tied to use only one version of Python/Pyodide (the one that Workerd embeds)
  * Package resolution is quite hacky and tied to workerd. Only precompiled "native packages" will be allowed to be used at runtime (eg. using a specific version of numpy will turn to be challenging)
  * Architecturally tied to the JS/v8 world, which may show some challenges as they aim to reduce cold start times (in my opinion, it will be quite hard for them to achieve <100ms startup time with their current architecture).

In any case, I welcome this initiative with my open hands and look forward all the cool apps that people will now build with this!

[1] https://pyodide.org/

[2] https://github.com/cloudflare/workerd/blob/main/docs/pyodide...

[3] https://github.com/cloudflare/workerd/pull/1875

Edit: updated wording from "proof of concept" to "release" to reflect the clarification from the Cloudflare team

This is awesome. I'd love to have upstream support in Wasmer ( https://wasmer.io )

This is awesome Nils! So happy to see the progress on the project since we chatted on the AGI house :) (I'm Syrus, from Wasmer)

dockerc works by using: Zig + crun + squashfs/overlayfs. Nils (the author) posted a bit more in this thread for anyone interested: https://news.ycombinator.com/item?id=39621573

Two thoughts after reading the article:

1. The post depicts properly the need of ownership from Elon Musk, but it misses the point of the Open Source Mission that OpenAI had when being created. And therfore, it misses the main point of the lawsuit.

2. It feels like the writing is assisted by OpenAI itself.

Am I the only one thinking that the strategy of the post (and maybe the contents) are completely guided by their algorithm?

This is super cool! It seems that endb uses emscripten to compile to WebAssembly, I wonder if they tried WASI or WASIX (https://wasix.org) to target both browser and server environments!

Indeed. Or using WebAssembly. In fact, Wasm cold startup times beat Javascript v8 isolate startup times by a significant margin!

I would love to see Wasmer Edge in the next comparison!

A summary for the lazy readers:

  * Cloudflare workers outperforms the rest by a big margin (~80ms avg)
  * Fly.io cold starts are not great for the Hono use case (~1.5s avg)
  * Koyeb wraps the requests behind Cloudflare to optimize latency (150ms best case)
  * Railway localized the app in one region (80ms best case, ~400ms rest)
  * Render has some challenges scaling from cold (~600ms avg)

In my opinion, this shows that all platform providers that uses Docker containers under the hood (Fly, Koyeb, Railway, Render) only achieve good cold starts by never shutting down the app. The ones that they do, can only achieve ~600ms startup times at best.

hahaha as a founder working on the Wasm space, this made me giggle!

Hey,

At Wasmer we are working on making any traditional app serverless-like.

Basically, we can freeze and restore workloads (work in progress, not yet released publicly) very easily thanks to WebAssembly and the Wasmer runtime. Because of that, the boot time is almost non-existing.

Let me know if you would like to trial it further and I can guide through the process: https://wasmer.io/products/edge

My email: syrus@wasmer.io

It seems they just added the mention to QuickJS, I assume, based on your feedback:

https://github.com/awslabs/llrt/commit/054aefc4d8486f738ed3a...

Props to them on the quick fix!