Run Python Bytecode in Your Browser | Hacker News Reader

Run Python Bytecode in Your Browser

github.com

91 pointsjayfk15 comments

As someone who was at the pycon au sprints watching this come together and hearing Russell talk about the potential uses that motivated him, I'm excited to see this continue. I'm hoping to find a little time to try and work on it in the coming week.

Pypy.js has demonstrated Python JavaScrip interoperability and DOM integration, so there's no reason this project can't have similar features. We can ship useful chunks of server side Python to the browser and stop duplicating our efforts.

Another way to do this would be to use numba package to generate LLVM IR from Python bytecode (it supports a rich set of instructions including numpy stuff, has type inference, etc), and then run it in the browser via emscripten, with some adjustments to the IR if required.

Quick googling yields a related blog post: http://cyrille.rossant.net/numpy-browser-llvm/

Numba is not a general-purpose Python compiler, and does not handle (or intend to handle) many basic Python statements (like class or function definition) at all: http://numba.pydata.org/numba-doc/0.20.0/reference/pysupport...

And most general-purpose Python (e.g. dicts, most builtins, etc...) require running in "object mode" which is not standalone, it calls into the CPython API and thus requires an underlying CPython runtime.

So no, numba is not an other way to do that. shedskin[0] might be though.

[0] https://github.com/shedskin/shedskin/

Actually many more general constructs like lists and class definitions are on the roadmap or in open PRs.

Point remains that Numba doesn't intend to be a complete general-purpose Python compiler.

The documentation is pretty minimal here, so I'm wondering if this has anything to do with WebAssembly[1].

If not, wouldn't it make more sense to target WebAssembly?

1. http://arstechnica.com/information-technology/2015/06/the-we...

The goal is being able to pickle something like a Django template and then run it client side. It's based on past work that has shown its possible to build bytecode interpreters without much code. It's not as much about targets and cross compilation or anything like coffeescript. It's about trying to run existing Python bytecode with as little code as possible.

I'm not sure I was clear or understand what you're saying. It sounds like you mean this:

Python source -> Python bytecode -> Python bytecode interpreter -> JavaScript engine -> browser

And what I meant was this:

Python source -> WebAssembly bytecode -> browser

Does that make sense? It's not like there's a bunch of Python bytecode sitting around that we can't possibly re-compile. It makes a ton of sense to create a Python-to-WebAssembly compiler, but very little sense to spend time adding functionality to a browser that's specific to Python bytecode.

> wouldn't it make more sense to target WebAssembly?

I don't think it would. A Python source -> WebAssembly bytecode compiler needs updates with every enhancement to the Python language.

An engine for running Python bytecode in the browser only needs updates with change to Python bytecode, which is presumably more stable than the language -- all the language-side work can be shared with the mainline CPython implementation.

We (PLASMA @ UMass, http://plasma.cs.umass.edu) have a project with similar aims that is somewhat further along: Ninia (https://github.com/plasma-umass/Ninia). Ninia is a Python bytecode interpreter in TypeScript. It is also being integrated with Doppio, a runtime system for running "conventional" programs inside the browser (https://github.com/plasma-umass/doppio), also from our group. DoppioJVM uses Doppio to run JVM bytecode in the browser (http://www.doppiojvm.org/). For technical details on both of these, see our PLDI 2014 paper: "Doppio: Breaking the Browser Language Barrier", by John Vilk (https://jvilk.com/) and Emery D. Berger (http://emeryberger.com): https://dl.acm.org/citation.cfm?id=2594293 (@ ACM), http://people.cs.umass.edu/~emery/pubs/doppio.pdf (non-paywall), video presentation here: https://vimeo.com/106106738.

Abstract:

Web browsers have become a de facto universal operating system, and JavaScript its instruction set. Unfortunately, running other languages in the browser is not generally possible. Translation to JavaScript is not enough because browsers are a hostile environment for other languages. Previous approaches are either non-portable or require extensive modifications for programs to work in a browser.

This paper presents Doppio, a JavaScript-based runtime system that makes it possible to run unaltered applications written in general-purpose languages directly inside the browser. Doppio provides a wide range of runtime services, including a file system that enables local and external (cloud-based) storage, an unmanaged heap, sockets, blocking I/O, and multiple threads. We demonstrate Doppio's usefulness with two case studies: we extend Emscripten with DOPPIO, letting it run an unmodified C++ application in the browser with full functionality, and present DoppioJVM, an interpreter that runs unmodified JVM programs directly in the browser. While substantially slower than a native JVM (between 24× and 42× slower on CPU-intensive benchmarks in Google Chrome), DoppioJVM makes it feasible to directly reuse existing, non compute-intensive code.

Looks like a version of the paper without the ACM paywall is at https://web.cs.umass.edu/publication/docs/2013/UM-CS-2013-02....

That's actually a much older version that is hopelessly out of date. I'll add a non paywall link.

http://people.cs.umass.edu/~emery/pubs/doppio.pdf

This is excellent, I'm definitely going to read through this and see if anything can be borrowed to help improve Batavia from these papers.

You can also chat with me (John Vilk), the lead grad student/dev on Ninia/Doppio/etc. I'm happy to answer any questions you may have!

Reply on news.ycombinator.com