I moved away from Poetry for Python

Before I found poetry I worked at a place that had python projects that were too complicated for pip to download dependencies for. I had developed a tool that, for our projects, could build a set of wheels to deploy a project. I had thought about what a general solution would look like but before implementing anything I found poetry.

Poetry struck me as one of those 80% solutions that might get in the way of a 100% solution. I used it for a number of little projects and felt it was good enough but my analysis was that its dependency resolution strategy was pretty slow and that I didn't believe it was 100% correct.

For my own Python projects I've been really slumming it lately. I make venv's with pycharm and let pycharm manage my dependencies. I also have written a number of "single file Python" programs that I use on my Linux server to do things like burn DTS CD-Rs with complete CD-Text information. For those ones I just stick to the standard library. One of these days I'm going to have to deploy a real python project to a server and I'll have to think of something then.

I convinced myself that the right way to resolve dependencies in Python is to do the same thing maven does and form a graph of the dependencies of all the software versions that could possibly satisfy the constraints.

For wheel files it is practical to do because you can download the metadata in 2 or 3 http range requests out of a wheel. eggs are a problem because you are expected to download and run the python scripts inside the egg before you know anything about the egg. fortunately (1) wheels have mostly replaced eggs and (2) you can build wheels as to the needs of your organization in a private "wheelhouse" used for builds in your organization.

Since it takes a long time to fetch that information it really does have to have a local cache like maven and maybe even a metadata database. It would be nice to see python build cleaned up.

I know there's a large part of the Python community that loves Poetry but I've never understood why. It adds a lot of complexity and I've never found it does a good job of solving complex problems, where as requirements and constraints offered by Pip are simple and when you have complex problems provide just enough (and no more) functionality to solve them.

For very simply requirements you can just have a simple requirements file or setup.cfg that lists what packages you need and use pip install.

As your requirements get bigger you can start putting sensible lower bounds on everything. This can be achieved manually or if you're struggling to extend an existing environment run pip freeze and replace all the == with >=.

When you start wanting to have reproducible environments or you're having to keep multiple requirements in sync with each other such as prod requirements and development requirements you use a constraints file. Install all your requirements at once, run pip freeze and save the output as a constraints file, now always specify -c constraints.txt when you pip install in any environment unless you are updating the dependencies in your environment.

Better still, you want to add 1 package without changing an of your other dependencies, just run pip install new_package -c constraints.txt and you either install it or find out what it's not compatible with.

It's not perfect but in my experience other tools have always seemed to add a lot of complexity but only solve a narrow set of problems, which I can usually just workaround in some other way with Pip.

This doesn't look good for the future of Poetry. Its maintainers are acting as obstinate as pipenv's. I guess I'll be sticking to pip-tools.

Main thing I want is lock files. I want to know exactly when full dependency closure changes and to see diffs when it does (and be able to reproduce why closure changed). Otherwise you get weird issues with packages updating when not expected. I was surprised linked James Bennett post didn’t mention that at all.

I’ve liked poetry in general, just takes a really long time and sometimes its solver gets stuck on a bad package.

I recently developed a tox plug-in, tox-pin-deps [1], that integrates the pip-compile workflow with dependencies specified in tox.ini (per environment). This was in response to both poetry and Pipenv being non-standard and consistent stumbling blocks for ramping up new devs on the team.

Despite the shiny, I seem to always find my way back to standard tools and text files.

[1]: https://github.com/masenf/tox-pin-deps

Poetry was the best pragmatic tool for python at one point, but it certainly was the developers project. That is to say, it introduced different and unfamiliar version specifiers to python, these are common place in other ecosystems, but this definitely was one of the barriers to success. Another came down to the applications code style and architecture, poetry is not written well, it tried to reinvent everything from the ground up in python package management, including it's own dependency resolution algorithm and cli library. I never could use it in isolation either, always had to combine it with other tools like Make. I once looked to contribute to poetry, it didn't take long for me to realise that it was poorly written and most of the new development efforts (e.g. the plugin system) were slowly developed by the project creator seemingly in isolation, meanwhile the rest of the development on the project slowly ground to a snails pace, at one point recently no new version (major, minor or patch) had been released in almost 18 months while the bug reports, issues and PRs stacked up, none addressed. Some of the most egregious issues were bugs in the private package repository support, some providers just wouldn't work and this can be a mandatory requirement for some organisations. One of the worst and most damning things about poetry, when it was actually releasing updates, was the regressions, it happened every single time, without fail, you only need to search GitHub issues for the proof. There are so many things that it could do that it does not, probably due to a poorly written foundation, a lack of leadership or organisation, almost definitely all of the above. Now it's just a tool that half works and has created a lot of animosity towards tools attempting to right it's wrongs. More than anything I am disappointed, I struggle to put my frustrations and whole feelings here, this is really what failure looks like from a thousand angles.

Recently I've ventured out to pdm. It is truly a sublime experience. It is everything poetry should have been and with a mere fraction of development team and effort, it really shows how much of a waste poetry is. Pdm has given me hope again. I only wish that more tools like Pycharm would support it, but really the experience isn't missing much at all. For those of you who are feeling the pain of poetry, I can't convey enough how much more pdm is, it's not perfect but if you try it in earnest and you won't be disappointed.

Somehow I became the “poetry guy” at work, and I spend a meaningful amount of time debugging shit for people because the environment somehow got corrupted or something. If there’s no issues I guess it’s fine (dependency resolution time notwithstanding), but when you experience a persistent bug or environment issue, it takes nothing short of expertise to figure out how to sort it out without losing half a day.

Switch to ruby and have all of the prototyping capabilities, way more fun, less surprising behaviour, and none of the package management shenanigans.

Been using pip-tools for years - it seems to just solve the problem in a clean and simple way.

How did Poetry get so much momentum vs pip-tools and pipenv?

I wish they had named it Pyetry instead of Poetry.