The Python scientific stack, compiled to WebAssembly | Hacker News Reader

The Python scientific stack, compiled to WebAssembly

github.com

173 pointstosh52 comments

Part of an effort to make a Jupytr notebook-like environment without relying on server side code.

https://github.com/iodide-project/iodide

> "Part of an effort to make a Jupytr notebook-like environment without relying on server side code."

So there won't be any server side code, you'd just be downloading the entire Python stack and dependencies every time you want to use this through your browser. This time it will be WASM though, instead of x86/x64/pyc.

That we can, doesn't mean that we should.

I guess we need an in-browser package management like npm?

Just compile Debian to WASM and use apt.

Just run a browser in your WASM Debian to hit the same page.

In-browser package management, combined with caching of common dependencies, yes.

Like npm? No.

Well I'm not really into JS web development, but NPM and a bunch of Node-like extensions would give you compatibility with lots of existing libraries.

> I guess we need an in-browser package management like npm?

It exists and it's called bower.

The plan is to build something based on conda.

That would be dumb, so why assume that would happen? You don’t download every picture on every refresh.

It would work on a Chromebook and a iPad also.

And yes I could pay for a Jupiter notebook hosted on some instance but I dont want to.

Maybe, but it might require more RAM and CPU horsepower than they provide in order to be usable.

Well, you'd download the entire stack indeed - but not every time. Your browser does caching too.

Plus, if it’s packaged as a PWA, it can run offline and seamlessly update the runtime in the background using a Web Worker.

It doesn't share the cache between the sites though. A local installation has only one runtime.

Absolutely; it's pretty much like installing an application that includes all its dependencies. Which, interestingly, is also the direction installed apps are moving towards somewhat, at least on Linux.

I can think of several scenarios where this could be useful.

There's an advantage even if you're running it locally. Now you can just load up some static HTML + JS files in your browser instead of spinning up a server and then pointing your browser at localhost.

As long as it's framed properly it can be acceptable. Ie. You get to a page with details and a "load environment" button with progress bars and all.

Of course when it's like 50-100MB you wouldn't use it in some cases. But once cached you're golden for your repeat visitor audience.

I benchmarked this recently. It comes in at 4-10x slower than cpython for random typical use cases on my machine, which is not too shabby at all. The author also states that numpy is slower than it should be due to lack of support for BLAS at this time.

The real impressive feat to me isn't hosting jupyter in the browser. It's access to a reasonably fast implementation of numpy in the browser which smokes native JS code for homogeneous array operations. I would love to see a minimalistic WASM implementation of numpy that can seamlessly interop with normal JS. Such a library would open up all sorts of possibilities that aren't currently feasible due to perf reasons.

I can't believe I'm saying this but it would be neat to have a browser native BLAS, or absent that perhaps cross-compiled to WASM from a RUST implementation or something! Makes me think of all the demos like the tensorflow playground (dunno if all that happens in the browser or not).

People are working on the pure JS version of BLAS: https://github.com/jacobbogers/blasjs . Hopefully that work leads to best practices and a standard

Interesting, how do you derive at "4-10x slower"?

Running the exported notebook here takes

time py3 python.py real 0m0.145s

chrome: load html: 2sec run html: 13sec!

That's not 10 times slower, but a 100 times slower!

Still good start.

I have used Weblas "GPU Powered BLAS for Browsers" for matrix multiply and it smokes! https://github.com/waylonflinn/weblas

Did you use js typed arrays (e.g. Uint32Array) to benchmark large array operations or just Array?

Numpy is fast thanks to having much of its code compiled as a native library.

They were purposefully working to reduce abstraction, since performance mattered from day one.

WASM is a huge step backwards in that regard. If you care about performance, just install Python on your box and "pip install numpy". Do you absolutely have to have it come in a browser now?

This is great. Many times I wanted to showcase to colleagues who don't use Python how things would work in Python, but they don't have either Python or Jupyter installed. So I had to run Jupyter server in my own environment and give them access to it. This is quite an inconvenience as well as a security risk. Having a completely server-less Python/Jupyter environment should make this kind of showcasing much easier.

You could also use a Jupyter notebook uploaded to colab.research.google.com

I'll chime in to mention notebooks.azure.com as well.

(Typical disclaimer, I'm a dev on the azure notebooks team; we just try to solve exactly the scenario GP was asking so I would be remiss to not throw our hat in the ring)

Why not show them the basics in https://repl.it/ or something similar?

Do they have Docker? Jupyter maintains a few docker images with various parts of the scientific stack.

How and why would they have docker and not python. Docker's like 10 rungs up the skill ladder.

If they're working with a different stack that includes docker?

Coming soon [...] Plotting using D3 from Python

Wouldn't Bokeh make more sense? I'd be very interested in a serverless Bokeh that didn't force me to ditch Python, for example.

Bokeh could probably be done, but most of Bokeh's client/server complexity becomes unnecessary in this execution model.

More recently, pyodide has grown full matplotlib support. See, for example, https://iodide.io/pyodide-demo/matplotlib-sideload.html?side...

What happens to all of the C code in Numpy, and the calls to external libraries like BLAS?

In particular, does this mean Javascript has access to Numpy? Sounds unbelievable, but would be extremely cool.

Yep. You can "import" python objects over to the Javascript side and start using them from there. It's not as pleasant as working with Numpy in Python, of course, due to lack of operator overloading, of course.

C can compile to WASM, but I have no clue about external tools like BLAS. I'd assume that things like Numpy are able to work without tools like BLAS, but you might also be able to compile the essential ones to WASM as well.

NumPy can work without libraries like BLAS or LAPACK, but it will be sloooooooooow. There are C versions of those libraries however, maybe they are using those?

Yes. It uses the C implementations of BLAS and LAPACK that ship in the Numpy source tree.

Python C extensions like NumPy are compiled into shared libraries, and loaded using dlopen() etc. normally by the main compiled Python runtime. (The only special thing here is that all those components are compiled to WebAssembly so it can run on the Web.)

I do a lot of work with Skulpt, which included adding (somewhat superficial) support for MatPlotLib. You can make line plots, scatter plots, and histograms. It's a pretty cool system, though I will say developing in it isn't always very fun :)

The goal of http://www.graalvm.org/docs/why-graal/#for-ruby-r-or-python is to run scipy.

Is there a performance hit compared to "native" Python?

Without a doubt.

There's a chart and some discussion here: http://droettboom.com/blog/2018/04/04/python-in-the-browser/ (and some followup at http://droettboom.com/blog/2018/04/11/profiling-webassembly/ )

TL;DR: pyodide in Firefox was slower than cpython.

Thanks. Note that the author of that post said:

> UPDATE 2018-04-11: My hunch was wrong, and I was able to get to the bottom of the root cause and significantly speed up these benchmarks.

And there's an updated graph here:

http://droettboom.com/blog/2018/04/11/profiling-webassembly/

Certainly for anything using Numpy.

Actually, Numpy-using-code is closer to native speeds than pure Python is.

The thing that seems to cause a greater gap between wasm and native speeds is lots of Python-level function calls, not the tight C loops that make up much of Numpy.

Huh I am getting

``` TypeError: window[s] is undefined ```

with firefox.

File a bug over at https://github.com/iodide-project/pyodide/

I'd love to get to the bottom of that.

Reply on news.ycombinator.com