Comment by acidbaseextract | Hacker News Reader

Comment by acidbaseextract | Hacker News Reader

Parent Full thread

acidbaseextract

The most frustrating thing is that pip doesn't make it easy to use more loose declared dependencies while freezing to actual concrete dependencies for deployment. Everybody rolls their own.

> Python libraries should not pin dependencies. _Applications_ can pin dependencies, including all recursive dependencies of their libraries.

Is the pypi package awscli an application or a library?

poetry is frustrating in that it doesn't allow you to override a library's declared requirements to break conflicts. They refuse to add support [1][2] for the feature too. awscli for example causes huge package conflict issues that make poetry unusable. It's almost impossible not to run into a requirement conflict with awscli if you're using a broad set of packages, even though awscli will operate happily with a more broad set of requirements than it declares.

[1] https://github.com/python-poetry/poetry/issues/697

[2] https://github.com/python-poetry/poetry/issues/697#issuecomm...

For this purpose, I’m defining a “library” as any PyPI package that you expect to be able to install alongside other PyPI packages. This includes some counterintuitive ones like mypy, which needs to extract types from packages in the same environment as the code it’s checking.

The awscli documentation recommends installing it into its own virtualenv, in which case pinned dependencies may be reasonable. There are tools like pipx to automate that.

Though in practice, there are reasons that installing applications into their own virtualenv might be inconvenient, inefficient, or impossible. And even when it’s possible, it still comes with the risk of missing security updates unless upstream is doing a really good job of staying on top of them.

I don’t think that respecting declared dependency bounds is a Poetry bug. Pip respects them too (at least as of 20.3, which enables the new resolver by default: https://pip.pypa.io/en/latest/user_guide/#changes-to-the-pip...). If a package declares unhelpful bounds, the package should be fixed. (And yes, that means its maintainer might have to deal with some extra issues being filed—that’s part of the job.)

Why on earth would you ever add awscli as a dependency? That makes very little sense. It’s an application (that is no longer distributed via pypi).

You should use boto3

> Is the pypi package awscli an application or a library?

Hopefully a library! As hopefully the AWS command-line interface is maintained and distributed separately from any SDK that powers it...

boto core/boto/boto 3 are the libraries, which awscli drives.

Reply on news.ycombinator.com