mattip

You're welcome, thanks for sharing. Do you have any numbers about speed vs. an alternative?

There is significant cooperation between the two. When porting PyPy to a new Python version, we examine what changes CPython made. And CPython core developers are aware of PyPy. The main obstacle is developer time. CPython has to be very careful about backward compatibility, which includes not making the interpreter slower for a few while faster for many.

You can comment on our blog, or open an issue. Frankly, we get so little feedback that dealing with new issues is not a hassle.

Cool. Yes, I am interested in hearing more.

Cool. Would love to hear more about the successes and problems, or even get a guest blog post on https://www.pypy.org/blog/

Lots of questions :)

For c-extensions see https://www.pypy.org/posts/2018/09/inside-cpyext-why-emulati...

We would like to be able to "just JIT" better. But for that we need feedback about what is still unreasonably slow, and resources to work on improving it. Right now PyPy is on a shoe-string budget of volunteers.

For rust, like CPython, use PyO3, which works with PyPy.

I am not sure about Pydantic. Sounds like a topic for someone to investigate on their codebase and tell us how PyPy does.

Thanks, that should have been https://mssun.github.io/rpython-by-example/index.html. Fixing.

Thanks for the feedback. It does seem like parsing logs and simulations is a sweet spot for PyPy

Interesting. I was under the impression PyPy did not do so well with SymPy because the dynamic code paths are difficult to JIT. What kind tasks waw a speed up?

It is much easier to embed CPython, PyPy can only be embedded via CFFI [0].

[0] https://cffi.readthedocs.io/en/latest/embedding.html

Python 3.10 is too old for your work?

You're welcome.

> that rely on incompatible extensions.

Which ones? Is using conda an option, we have more luck getting binary packages into their build pipelines than getting projects to build wheels for PyPI

It supports Python3.10 now too. Thanks, I updated the site.

> it's probably all about string index into dict and dict management

Cool. Is the performance here something you would like to pursue? If so could you open an issue [0] with some kind of reproducer?

[0] https://foss.heptapod.net/pypy/pypy/-/issues

Thanks, that is hopeful, although quite a while ago.

Is your group still using it?

Here ya go. On these sometimes one is faster, sometimes the other. https://github.com/kostya/jit-benchmarks/blob/master/README.... Personally i don’t like such comparisons. Benchmarking is hard and far from objective. Much of what makes python popular is the developer experience. Generic benchmarks will only give a rough guide about what to expect in your application. If you are in a niche like the OP, you will have to figure out how to handle your bottlenecks.

I am glad you find PyPy useful. The good news is exactly the boring consistency with which we have been able, on a volunteer basis, to stay pretty close to CPython progress. There are no big challenges there, it is work at the top layer of the interpreter to adapt current code to CPython changes. On the other hand, we need sponsorship to implement the deeper changes we would like to make that would make things even faster.

PyPy is still around. We released a Python3.10 version last month. What else would you like to hear about?

There is heptaood, https://heptapod.net/, a friendly fork of gitlab that supports mercurial. PyPy and others are using the free instance at https://foss.heptapod.net/public

Sure but this begs the point of what happens if the underlying ISA model changes. For both the C and RPython generated emulators, they build from source so they can regenerate in minimal developer time. For the Spice or qemu based emulators, you need to do much more work. So the Sail based tools are targeting a different problem space than the Spice or qemu based ones.

There is a general parallel here with Python vs. compiled code. Many developers prefer the flexibility that Python gives them over the speed that compiled code gives. Once the problem space is well defined and static, or small enough to be well understood, it makes sense to spend the developer time to create a high-performant solution. But during the exploration phase, it is better to use a dynamic language that may be slower.

RPython is a limited Python subset that can be compiled to C. It provides a meta-JIT and a garbage collector for interpreters built on top of it.

While RPython is a restricted version of python used to build the PyPy python interpreters, the interpreters themselves are not restricted. Any deviation from CPython behavior, intended or not, is considered a bug. So Codon should be compared to the PyPy python interpreter, not to RPython. The advantage to writing the interpreter in RPython rather than C (CPython) or pre-compiling python code to LLVM IR and from there to creating and executable (Codon), is that RPython comes with a metaJIT (which can generate a JIT) and a mark-and-sweep garbage collector for any interpreter built on top of it.

> You see this with Opensource too

I think it depends what project you want to interact with. In the projects I am involved in (Python, Numpy, SciPy, Cython, PyPy) you will get a response from a core dev quite quickly.

Isn’t number of commits a very weak measure of activity? All these metrics depend greatly on the size and maturity of a project, the base language the majority of code is written in, and the culture of the company or community behind the project. I would tend to use them to look at a single project over time rather than between projects.

I don’t really know, sorry. There are a lot of moving pieces as the company grows (we are hiring) and the interplay between the pure consulting, open source work, growing the Venture fund is dynamic.

PostgreSQL has a number of companies providing support, the db is free and open source. Ray is offered for free, Anyscale provides support. Quansight offers qhub for free and provides support. Those are just a few off the top of my head. Disclaimer: I work at Quansight and contribute to ray.

This project wraps py-heat https://github.com/csurfer/pyheat in a convenient to use ipython magic command. Pyheat presents a heat map of profiler output: i.e. the output of running your program and measuring the time to execute each line of python code. Pyheat in turn uses pprofile https://github.com/vpelletier/pprofile.

A bit if a generalisation. For a different view: currently almost all the core devs for NumPy and SciPy (those are the ones I know personally) have doctorates.

Right, Sphinx itself is not JIT friendly, but I thing django should be faster