Wasmer: A Python library for executing WebAssembly binaries | Hacker News Reader

Wasmer: A Python library for executing WebAssembly binaries

github.com

167 pointspplonski8621 comments

Looks good! Just for my understanding, what would be the advantage of working with wasm binaries instead of, let's say, C external libraries or DLLs via ctypes?

I can think of: 1. Portability of the wasm binaries

Is easier interoperability between data types an advantage as well? Ctypes can be a nightmare when passing around structs and pointers (and pointers of pointers, etc.)

I have an actual real-world use case for this :)

Shader-cross-compilation for 3D frameworks:

https://floooh.github.io/2017/05/15/oryol-spirv.html

I'm hooking python scripts as custom-build-jobs into C/C++ projects which (for instance) takes care of compiling GLSL shader code into HLSL (D3D), MSL (Metal) and various GLSL versions (for GLES2, GLES3 and desktop GL).

Khronos provides C++ projects which help with this stuff:

https://github.com/KhronosGroup/glslang

https://github.com/KhronosGroup/SPIRV-Cross

So I'm using Python to hook my custom build jobs into the C/C++ build system (via cmake), but I still need to invoke native command line tools (or DLLs) which wrap around those Khronos libraries. Currently this means building (at least) 3 executables for Windows, macOS and Linux, and "distribute" those precompiled exes with my python code.

It looks like with this python+wasmer solution, I can instead compile the Khronos libraries into WASM modules, and the result runs "anywhere" where wasmer runs (e.g. also Raspberry Pi, BSDs etc...).

That's really cool stuff :)

There's also sandboxing. You shouldn't load untrusted DLL, but loading potentially untrusted WASM is a design goal of WebAssembly.

Portability might also bring along with it easier reproducible builds? Right now I believe if you use wheels, you basically have to trust that the distributed binary matches the source code. I saw a case where a package was taken over by a new maintainer who released wheels for an existing source release, and everyone basically shrugged and said, eh, this is probably fine, because there was no easy way to check. It sounds much easier to build an automated system that could check if wasmer-based packages are faithfully compiled from source, because there's only one compile target.

Sandboxing also sounds valuable, as mentioned by a sibling. I'm thinking of projects like imagemagick that run all kinds of untrusted input through all kinds of processing. It would be nice if there was a simple default way to write data processing libraries the way djb would write them.[1]

[1] https://cr.yp.to/qmail/qmailsec-20071101.pdf

My first impression is that the goal of this project is to run binaries that were likely built for other purposes like the web, not that they actually think that making libraries into webasm and running them into python has any particular advantages vs other options that already exist

This looks great, I like the simple examples which show how to get a string out from the web assembly module. One thing I didn't see in the readme or examples, however, is a way to import python functions to the web assembly modules. Is it possible, and just missing from the documentation atm?

We don't support that yet, but we are working on it. It's not that easy even if the theory is kind of simple to explain. The implementation in the different backends is mostly the challenge. The Python extension uses the Wasmer runtime (https://github.com/wasmerio/wasmer) which supports 3 backends to generate executable code: Singlepass, Cranelift, and LLVM.

Cranelift is the easiest one to embed, and it brings a great balance between compilation time and execution time. That's the one used in the Python extension right now.

The “import functions” feature is also missing in the PHP extension (https://github.com/wasmerio/php-ext-wasm), but yeah, as I said, we are working on it. It's coming soon!

Once you allow to call Python, keeping things sandboxed becomes pretty hard, doesn't it?

That's true. However, it allows to integrate your Wasm program to your favorite framework more easily. It's likely to be a small surface of code to review carefully. It's always a balance :-).

It's often for code that is asynchronous that will not return an answer immediately... the function passing is most often used to send a callback for these situations.

A state-machine, generator or coroutine can be used instead of callbacks.

I mostly prefer only having one direction of who-calls-who when combining multiple languages. Though I understand that for a general purpose system like Wasmer it makes sense to support everything, so that users can choose the architecture.

understood, thanks for the explanation - keep up the great work! is there anything external contributors can do to help? :) I didn't find any issues labelled "good first issue" but I did find a related issue[0]

[0]: https://github.com/wasmerio/wasmer/issues/267

The best way to help right now is to test it, and report any issues! I've been able to try it on Windows so far for instance.

Also the API isn't complete. We need to be able to grow the memory for instance. We need to understand the community needs to define a roadmap for the community itself.

Looks like the current code doesn't support it.

https://github.com/wasmerio/python-ext-wasm/blob/master/src/...

but in theory it shouldn't be too hard™

This looks really fun to use for algorithmic trading (we have to deal with and sandbox client code not written by us here at KloudTrader). Wonder how hard it is to integrate with numpy etc.

I'm sure there are better ways to do this, but if you really want Python client code -> WASM -> Python host, I'd imagine you could probably run https://github.com/iodide-project/pyodide within Wasmer. But you'd likely lose a lot of the performance benefits of numpy etc. being compiled for the exact intrinsics supported by your host CPU. Container-level sandboxing still sounds like a superior bet.

Container level sandboxing is pretty difficult to do well without hypervisor/VM support.

This is awesome! I wanted this so much. Need to play with this asap.

"Safe: All calls to WebAssembly will be fast, but more importantly, completely safe and sandboxed."

That's a very bold statement, I see no backup of this in the readme.

It's a side effect of the design of WebAssembly, as long as wasmer itself has a conformant implementation (the WASM spec itself is already formally verified).

I recommend this excellent reading, https://00f.net/2018/11/25/webassembly-doesnt-make-unsafe-la.... It clarifies what “safe” means in this context.

Reply on news.ycombinator.com