Hacker News Reader

Comment by aprdm | Hacker News Reader

Argh, Python is moving too fast. 3.8 has barely settled in with the walrus operator.

For people doing docker based stuff with the latest and greatest always it might be OK, however, for enterprise environments where OS versions do not move as fast and you have runtime dependencies everywhere it isn't good.

We have to build C++ libs against python versions, have thousands of users having their workstation in a sane version of python globally where all dependencies (1k+ in-house/external packages) work well and etc. etc. (think CAD kind of software / Qt UIs)

We were still in the process of migrating towards python3.7 and it was a huge effort for barely any real gain. Hopefully new libraries do not use 3.8/3.9 only features otherwise we're in for a bad time..

View on HN

Python 3.8 will still be supported until 2024, that is, 5 years from its release. I guess that's fair enough.

> Hopefully new libraries do not use 3.8/3.9 only features

Ideally, for production environments, you'll want to pin version numbers for everything you can, so new packages don't intrude. I also like to keep testing environments with minimal dependencies and no pinning (I built `pip-chill` for that). Wouldn't that work for you?

That helps not to break an existing codebase.

But not if there's a nice new library you really have a need for that uses new 3.8 features, and you can't use it because you're stuck with 3.6 for example...

We can always offer a PR that allows the library to run on older versions. I am limited to 3.6 on many environments and would be extremely happy to know others on 3.6 have better library support.

Having said that, It's not something I feel. I know some things can get faster and some syntax can be improved, but, so far, I'm pretty OK with those envs running on 3.6.

As for me, locally I use 3.8. If I do any 3.8-ism, CI will catch it.

That works but doesn’t solve the problem mentioned by the other comment. We also have to support a matrix of Python runtimes and dependencies in all of our machines. We can’t pre-bake the environments. It’s a very unique problem for HPC I guess. We don’t use pip, have a look in the Rez package manager if you haven’t yet for an idea of the problem.

For our webservices it is much less of a problem as we can isolate the runtime from software installed in machines or third party software dependencies that we integrate with (think a blender plugin, blender comes with its own versions of dependencies)

> We don’t use pip

What's the problem?

Have you read about the problems Rez solves ? Our environments are too dynamic and need to always be available. Our hpc cluster runs millions of different combinations of software runtimes at any given day...

My understanding is that it operates on a different level than Pip.

Hm, not really. It has build scripts which you do whatever you want to build your package and put somewhere and then it adjusts PYTHONPATH dynamically at runtime.

You don't need Pip if you're using rez. You also don't use a setup.py, you use a package.py instead.

Rez looks really great! Thanks for sharing.

The problem is vulnerabilities. Your vulnerability scanner says you're pinned dependency has a vuln, and the only solution is to upgrade versions.

By and large, libraries don't backport vulnerability fixes. You have to update from 1.0.1 to 2.3.3 because they only were notified of the vuln after 2.3.2 was released. In practice, almost no open source libraries would also release a 1.0.2. Maybe some of the largest most well-run libraries will, but enough don't that you will run into this problem.

> In practice, almost no open source libraries would also release a 1.0.2.

Well, if package consumers such as yourself don't put in the legwork to make that happen then yeah, don't expect others to do the work you personally need for free.

I'm sure that if you posted a pull request for that hypothetical 1.0.2 release then the project would gladly receive it with open arms.

People should keep in mind that open source software isn't a one-way street.

I don't think the parent comment argues that anyone should backport their changes for free. Their point is, upgrading everything is cheaper than backporting security fixes.

Genuinely asking because you’ve described the sort of CI/CD problem that there are existing solutions to, things like the Open SUSE build service, red hat have their own tools, commercial services exist from several vendors designed to tackle these “operating system permutations” build problems... is one release a year with security fixes going back four versions... really too fast?

All the gory details about the release timeline are in PEP 602 - https://www.python.org/dev/peps/pep-0602/

I believe the biggest issue is having non compatible additions to the language... for example, any code written in 3.8 with walrus operator will not work in python3.7.

That is IMO not ideal. If it was security fixes / new features / improvements with no new syntax then I would be OK.

I would say a lot of companies still have a very big legacy code base in 2.7... new services are created in docker using the latest python3 but desktop applications aren't as easy to upgrade constantly.

Some companies also use NFS to store python libraries (we release 100s of times per day) and make some wrangling of PYTHONPATH to allow applications to find dependencies. Having to support multiple (incompatible) versions of python at a workstation isn't a good problem to have.

Backwards compatibility I understand, but which languages give forward compatibility?

I mean how would that with new features even work?

Hm not exactly what I tried to convey. I guess I would love if python stopped trying to add new syntax so often.

Walrus operator, fstring, etc. should happen maybe once every two years ideally? Did we really need the walrus operator? The cost of adding it is making any library using it in python 3.8+ incompatible with runtimes of python3.8<.

In the ideal world, library developers would properly use the python_requires[1] option and dependency resolvers would correctly identify the latest version of a library that works with the installed version of Python. Unfortunately I have found that is not always the case.

When I make public modules, I personally weigh whether using a new feature is worth losing users stuck on an older version. That sweet spot for me right now is targeting Python 3.6 and using backported modules where necessary (e.g. mypy-extensions). I'm a big type hint fan and type hinting became much more usable in >= 3.6.

[1] https://packaging.python.org/guides/distributing-packages-us...

>Backwards compatibility I understand, but which languages give forward compatibility?

Python used to offer it frequently. from __future__ import ...

Code written in C89 can trivially link against code written in C18. This also mostly works in C++ as long as new features are not used at the interface level and you don't have an ABI migmatch (e.g. with stdlibc++ std::string, although that's possible to work around with proper configuration). Any language exporting a C FFI should work just fine.

> We were still in the process of migrating towards python3.7 and it was a huge effort for barely any real gain.

Besides the pursuit for the latest and greatest, is there really anything forcing you to upgrade?

I mean, Python 3.7 will be around for at least 2 more years.

https://www.python.org/dev/peps/pep-0537/

The problem is the "huge effort", not the "barely any real gain": you are not automating enough. Building and packaging "1k+ in-house/external packages" should be longer than building only one, but not more difficult with proper scripting and build systems.

Yeah of course :) we have terabytes of Python code written over the past 20 years that support our business. Good luck justifying to the business to stop producing code that generates revenue to “modernize” all that stable code.

Are you migrating to 3.7 from 2.7, or 3.6?

I would say 90% of our code base is still in 2.7 and 10% in 3.6.

That’s for the desktop applications, we have a bunch of services in 3.8 already.

The Python packaging "solution" is to have a whole Python world for everything you want to run (aka insanity).

Heard of virtualenv?

Yes... that's exactly what I was referring to. The problem is this: https://github.com/pypa/packaging-problems/issues/328

I've been thinking of offering a patch to `fdupes` to turn duplicates into CoW copies. ZFS and BtrFS have them covered, but I'm not sure about APFS and I bet NTFS doesn't.

In any case, it's really hard to get a workstation with less than a terabyte of storage in it. I have 2.9 gigs under mine and that's for 53 environments. I'm not sure that's a huge issue.

Sure in some use cases it doesn't matter. In others (e.g. on an HPC cluster where you have a quota in your home directory measured in GB) it does. CoW won't help on GPFS or Lustre, probably. Arguably Python has no place in HPC but a lot of people use it. And on my laptop technically I have the space but I am loathe to give it up to duplicate various packages many times.

What would you suggest instead for HPC?

Well, generally you want something with better performance characteristics, which increasingly is not the case (see e.g. https://www.nature.com/articles/s41550-020-1208-y%C2%A0). Though, even if your stuff is implemented in Fortran or C or C++ or on a GPU, Python is still a natural scripting language that is often the primary interface to some other piece of code, thanks to swig or boost::python or cppyy. But Python's fast rate of change and lack of backwards compatibility makes it problematic for this use. Our HPC cluster recently forced everyone to upgrade to Python 3 and that caused all sorts of problems for everyone. I understand there isn't really a choice with the lack of Python 2.7 support, but this has not been a great experience for anybody.

Yeah totally. Our use cases seem very similar. We still haven't transitioned to python3 at our HPC.

Well.. we do have it available but only around ~10% of the workloads make use of python3 and their libraries. We have created the two different paths in our NFS and if you're in python3 then you can't access python2 and vice-versa.

We will be migrating everything to python3 eventually but as you noted it is a big effort with a lot of possibility for chaos.

It's somewhat dangerous, but you can always replace dupes with links that work in your filesystem. Lustre, IIRC, supports hard links. I'm not sure whether GPFS does that, but I don't see any reason for that. If that fails, symlinks should work. The Python environment directories shouldn't be writable anyway.

> The Python environment directories shouldn't be writable anyway.

Yeah, except people start putting together their own Python environments in their home directories because that's what third-party packages suggest and asking admins to install random packages globally for random one-off tasks is not really sustainable. I'm sure there are workarounds, but it would be nice if virtualenv had some automatic way of figuring this out.

Sorry. Not what I meant. Your programs shouldn't write to these directories while running anyway, so once deduplicated, they should remain that way.

Ah sorry! I see what you mean now. I guess it wouldn't be so hard to write a script to deduplicate a bunch of virtualenv environments, but I'm scared to be the one to go down that rabbit hole.

There is a version of fdupes that makes hard links. Maybe it's enough for what you need.

Yes, and snap/flatpak's idea of software installation is a whole container per software.

You should choose one: Isolated environments or dependency hell. Each has its own pros and cons.

Reply on news.ycombinator.com