Hacker News Reader

Comment by wjakob | Hacker News Reader

Caution: if you rely on pybind11 or a project using pybind11 (many projects do, like NumPy/SciPy/Tensorflow/PyTorch..), hold off on upgrading to Python 3.9.0 for now.

A change in Python 3.9.0 introduces undefined behavior in combination with pybind11 (rarely occurring crashes, but could be arbitrarily bad). We will work around it in an upcoming version of pybind11, and Python will separately also fix this problem in 3.9.1 slated for release in December.

Details available here: https://pybind11.readthedocs.io/en/latest and https://github.com/python/cpython/pull/22670.

View on HN

I was about to ask why would they do a release if they know it's broken and blowing up their flagship libraries? It's worrying when the interpreter starts to favor shipping quickly over being functional.

But nevermind, the 3.9 release was 15 days ago and the bug was identified 8 days ago. I guess there has to be a release before people will start using it and find bugs/regressions.

There's a saying to always wait for the .1 release, this is a good illustration of why.

What makes pybind a "flagship library"? If it was very widely used this bug probably would've been caught by people testing the alpha/beta/rc versions of 3.9.

People using numpy are not in the fringe. If this wasn’t triaged during prereleases then it’s either that it doesn’t always cause problems, or there is an issue in the diversity of testers.

> If it was very widely used this bug probably would've been caught

As the top-level comment says, it rarely causes crashes.

> What makes pybind a "flagship library"?

NumPy, SciPy, and friends are, individually and collectively flagship libraries, and depend on pibind11.

> If it was very widely used this bug probably would've been caught by people testing the alpha/beta/rc versions of 3.9.

They are very widely used, but “probably” means “do enough releases and you will find exceptions”. Welcome to an exception.

the parent was probably talking about NumPy/SciPy/Tensorflow/PyTorch, which I would also label as flagship libraries...

I personally don't like Python (I prefer Julia/Ruby/Rust), but all the new advances in machine learning are implemented in Tensorflow and PyTorch, so I came back to Python just because of these libraries and infrastructure on top of them.

I was just using the expit function from scipy yesterday by the way, because that's where it was implemented.

I don't know if I would call the SciPy ecosystem the flagship libraries of python. I know it is very popular,and probably in the top 5 use cases, and I use some of them everyday. but in my experience they've always been off in their own world in relation to the rest of the python community. Most of the people I've met that use this stuff are either scientists or finance people that are just using it as another tool along with matlab, R, and excel. Which is what they should be doing, but in their minds they're not writing python code they're using Pandas, Jupyter, and Matplotlib.

But yes, never ever jump on a new release of anything, at least in production. Always stand back and wait for someone more optimistic to find the bugs

> I don't know if I would call the SciPy ecosystem the flagship libraries of python.

I mean, sure NumPy is integral to python finance and scientific computing...but it's also right at the heart of the popular Python roguelike tutorial, too.

There's Python code that doesn't use NumPy, sure (I've got some in production right now), but I can't personally think of a Python library with nearly as deep dependencies across nearly as diverse a range of use cases.

hold off on upgrading to a new major version is always a best practice, unless you really need a specific feature in the new release

Or you feel the call to adventure!

Reply on news.ycombinator.com