Pyjion – A JIT for Python based upon CoreCLR | Hacker News Reader

Pyjion – A JIT for Python based upon CoreCLR

github.com

72 pointsLofkin23 comments

One of the most promissing projects :

  * it keeps CPython as is.
  * hence retains compat with extensions.
  * but still has the potential to improve performance like pypy.
  * and yet can be shipped just as an additional compiled lib inside the Python dist.

If it ever works, it will be wonderful.

It would also be nice if it worked outside of Windows.

I had the same though and looked into it. It seems that CoreCLR isn't the limitation there. It supports a variety of platforms: https://github.com/dotnet/coreclr#build-status

Looks like the authors are focused on Windows for their initial prototype development.

Yep. They are MS ingenieers, of course Windows is easier for them. Plus it's good publicity. Given that they are paying for it, it's fair they start like that.

Exactly, we started on Windows out of ease-of-start/momentum. There are absolutely no plans to make this a Windows-exclusive. Basically we want to show this work pans out before working before putting in the work to make it cross-platform (we have an open issue to move to CMake to help solve the cross-platform problem: https://github.com/Microsoft/Pyjion/issues/76).

Exciting work! Is pyjion exploratory or is MS determined to see this through?

Yes. :)

It's exploratory in the sense that we are seeing if this general approach works for CPython. But if this exploratory work shows promise, then we will see it through. And since I'm also a member of the Python development team I always want to see Python be faster and succeed more. :)

Sweet, new Microsoft is awesome!

Pyston seems to be catching up to PyPy fairly quickly ...not sure if there is anything transferable to pyjion, but if you haven't done so might be worth checking out.

Also for getting around the GIL, PyParallel seems to have a really interesting (and seemingly successful approach: http://pyparallel.org/

Its being worked on by one of the Guy's at continuum...and It's definitely in continuum's best interest to help python's future so I'm sure they wouldn't mind collab.

PyParallel and something like Pydjion are definitely complementary technologies; PyParallel focuses on exploiting things like async I/O and simultaneous multithreading, ensuring that the most optimal technique is being used to go from hardware to OS to Python.

Once we're back into Python, though, Pyjion's JIT'd version of whatever would kick in and do its thing. Perfect combo.

Trent already replied about PyParallels (and yes, we are aware of his work).

We are also aware of Pyston. There might be a chance that they could end up using our JIT hooks, but they also started a full year before us so they are not currently structured to plug into CPython like we're trying to through the API we're designing (although hopefully they will be able to, but who knows).

if the speedups pan out, that is the plan! the work on performance has not been started yet however.

Awesome!

A while back there was some talk of native excel-python integration. Any chance that is in the pipes, or is MS sticking to R in that sense?

You can vote for that feature at https://excel.uservoice.com/forums/304921-excel-for-windows-...

Sounds great. Looking forward to testing it out.

Here's their list of passing and failing CPython tests:

https://github.com/Microsoft/Pyjion/blob/master/Tests/python...

One thing to note about the list of failing tests is the vast majority of them are due to a single bug (or two): https://github.com/Microsoft/Pyjion/issues/103 . And we currently only have 11 open bugs so the test failures make it look worse than it is.

Thanks to all of the great questions we got, we have now written an FAQ for the project: https://github.com/Microsoft/Pyjion#faq

How does Pygion compare with IronPython?

From reading the description, and from what little I know about IronPython...

IronPython is a re-implementation of Python in C# which allows it to play nice with .NET. This is similar to Jython which is a re-implementation of Python in Java that plays nice with the JVM.

This project on the other had is not a re-implementation at all, but aims to augment the existing CPython implementation with JIT APIs and is using CoreCLR for the JIT.

What ericfrederich said is all accurate. We are also considering playing with ChakraCore as a JIT back-end as well once we are sure that our abstract interpreter code that drives the JIT emission code is compatible with Python semantics.

Woah!!! Cool

Is there a PEP for this, or is it too early, more of a proof of concept?

It's too early. We want to prove to ourselves that we can get the performance we want before proposing the changes we think we want to CPython's C API.

Reply on news.ycombinator.com