Hacker News Reader

Comment by twic | Hacker News Reader

> Python serves several huge ones: web, datascience, 3d graphics, sysadmin... all with different needs, different conventions, different “standard” tools to do this and that.

Them having different conventions and standard tools is the failure, not a constraint.

What are the different needs that these ecosystems have which mean they couldn't use a common package manager? I don't believe there are any.

View on HN

For instance, in the scientific computing world, compiled extensions are hugely important, whereas they're relatively rare in web development.

There's probably no logical reason why a single common package manager for these domains is impossible. But they have different priorities, and people write tools that solve their own problems. Who is going to decide which package managers are unnecessary, and convince all of the users who are happy with them to switch to something else?

> relatively rare in web development

not at all. from my previous experience with ruby i was using gems with compiled bindings to libraries for: mysql, memcached, json, xml, regexp and http. and probably some more i don't recall right now.

Python does all of that and more. There are still occasional issues with the thorniest setups, but binary wheels absolutely took care of the average case. You even get a massive desktop toolkit luke QT with a simple “pip install pyqt”.

Getting there might not be the nicest experience, but most packages nowadays can and do get there.

NPM has excellent support for compiled extensions, despite being mostly used for web development.

I think something people are forgetting when comparing Python's packaging system to other languages and runtimes is how old Python and its ecosystem is. For instance, Python's public release predates Linux's public release(Feb 1991 versus Aug 1991). The first release of setuptools was 2004 and it isn't like people weren't sharing Python code even before then (https://github.com/pypa/setuptools/commit/8423e1ed14ac1691c2...). That's over 5 years before npm was released (https://en.wikipedia.org/wiki/Npm_(software)). That means npm got to learn from Python and any other project that predated it. It also means they didn't have Python's momentum to contend with (I certainly never expected Python to become _this_ popular, and that makes it very hard to change as you literally have tens of millions of people to re-educate when you change something).

So while it's fine to say you wished Python's packaging approach was as good as another languages, please do so know that it is not fair to say that if one community got something to work for their needs it means Python should have been able to as well; it's not an apples-to-apples comparison considering the trees were quite possibly planted at different times and are different varietals.

The age also means Python has had plenty of time to come up with something new!

And it (and by "it" I mean the Python community) absolutely did! The main problem that it has come up with numerous different somethings that have been competing to become standards, across a wide range of related, but separate areas (packaging, versioning, environments...), and only a few of them (like pip) managed to become partial (but not at all universal) de facto standards.

I'm not sure how old you are, but if you don't know the name "Borland" you might not realize how novel it is to have a compiler on every major OS. While Python's age means it has been around to see a lot of changes occur, that also means it had pre-existing practices and code to also consider whenever a shift in the industry occurred. Plus in hindsight things look obvious, but during the transition it isn't obvious what will stick and what won't.

It's pretty remarkable that things have continued to function since my 17 years with Python alone have seen things like:

- Compilers becoming available on all OSs - Linux becoming a thing - Laptops becoming common - Smartphones - Web-based email clients

As I have said a few times in comments on this thread, it's fine to feel like there's room for improvement, but trying to "stick it" to Python and its packaging ecosystem in some way for not being perfect is pointless for you and demotivating for those who are actually in a position to try to make things better for you (and even if you aren't a Python user you should care to some extent about it as Instagram runs on Python and thus it is helping to deliver you those cat photos while you're stuck at home ).

I’ve been doing this for over 20 years. It’s no excuse, python should have a solution.

It sounds a bit more understandable if you rephrase it as "a small group of largely unpaid volunteers should find a solution for which millions of users and several billion-dollar businesses will happily abandon their current long-running and well-tested solutions".

First, programmers have choices, and Python's lack of a good user experience on this front will turn off people that have better options with other languages which have found a way to deliver a better experience given the same constraints as python.

Second, there is the python foundation, which funds people to work on areas of python. This should be a priority, and can be done by paid contributors.

People in the Python ecosystem have come up with plenty of new things over the years - eggs, wheels, conda, poetry... Coming up with something new is the easy bit. Getting everyone to agree that a new thing is better and stop using/recommending other things is much harder - almost impossible, if those other things serve some use cases fairly well.

The fragmented ecosystem is one of the main complaints about Python packaging today. And that's really tricky, because the main way the open source world knows to solve problems is to create new tools, and that doesn't help when the problem is 'too many tools'.

Working in 3D graphics there are lots of different constraints unfortunately and it isn't an easy solution.

A lot of 3d graphics houses have to roll their own solution.

This is a famous one in the cycle and a common approach-> https://github.com/nerdvegas/rez

The issue is that the environment varies a lot and there's usually a lot of C++ code that needs to be compiled to interact with the python plugins based on different environments with 500+ libraries mixing & matching...

Reply on news.ycombinator.com