Hacker News Reader

Comment by woodruffw | Hacker News Reader

> It's a bit ridiculous that setup.py allows you to run custom code on install.

Agreed! The good news is that it's slowly getting better: just yesterday[1], setuptools added support for PEP 660[2], which removes one of the last remaining reasons to use a `setup.py` instead of a fully declarative `pyproject.toml`.

Edit: Also, just to qualify: this is a consistent problem across packaging ecosystems. Even system package managers (reasonably!) allow custom code during installation, e.g. to add service hooks and update locale files.

[1]: https://twitter.com/juanluisback/status/1557734536586625025

[2]: https://peps.python.org/pep-0660/

View on HN

Still doable with pyproject.

I actually wrote a Python tool for downloading binary releases from GitHub and Gitea and wrote a “builder” module that can be used in a pyproject file to tell it to download an arbitrary binary and run some commands.

Here’s an example in action: https://github.com/IamTheFij/release-gitter/tree/main/sample...

Hopefully next they'll incentivize package maintainers to migrate to the declarative way.

Perhaps spit a scary sounding security warning for all packages which use a setup.py?

> Perhaps spit a scary sounding security warning for all packages which use a setup.py?

This would be nice, but there's probably too many practical barriers to this. For example, you can't (easily) replace `setup.py`'s dynamism for packages that contain native packages: they need to be able to spawn compilers, etc. and do so in ways that are hard to describe declaratively.

We probably don't want to provide a security warning on every native non-wheel package, so that's a no-go for the time being.

Could we perhaps sandbox any such compilation steps within a container and then copy out the resulting artifact(s), with a hook that does some security scanning?

This is hard to do in the general case: the compilation steps often need system dependencies or to otherwise query system state to do the right thing, which means that the sandbox has to accurately reflect the host.

In terms of hooks, audit hooks and events[1] get us some of the way there. But they can be circumvented as well; I wouldn't rely on them for untrusted code.

[1]: https://docs.python.org/3/library/audit_events.html

Should start with a warning, and move after one year to users having confirm that they allow pqckqges to execute setup.py and ultimatrly not allow it any more.

This would probably break too much of the ecosystem, since many top Python packages include native extensions that can only be built with a dynamic `setup.py`.

Wheels are solving that problem, but it's likely going to be many years before warning about `setup.py` is reasonable and not disruptive to the majority of legitimate users.

Instead there's a scary deprecation warning when a package uses setup.cfg, which is another declarative replacement for setup.py.

And some pyproject.toml features are still experimental per the setuptools docs, so I'm not sure which method to choose for our internal projects.

I'm glad the setuptools/pip/Python devs are cleaning up the packaging situation though. pyproject.toml seems perfect, when it becomes ready.

I published my first Python code in a while using a pyproject.toml and it was amazingly simple. As someone who was doing Rust for a while I really feel like they're learning from the modern ecosystems very well.

Do you have a link to a good tutorial for this?

Follow the setuptools doc [1], it's very easy and there's an example you can start from. It links to PEP 621 [2] which seems to be the specification if that can help.

[1] https://setuptools.pypa.io/en/latest/userguide/pyproject_con...

[2] https://peps.python.org/pep-0621/

Is toml still not an included dependency in python?

There will be a `tomllib` in Python 3.11: https://docs.python.org/3.11/library/tomllib.html

Why the discrepancy between csv having the csv package but toml having the lib suffix

Not in 3.10. It's included in 3.11.

(But this doesn't matter for packaging purposes, since setuptools is also not included in the base distribution, but does include TOML support.)

Reply on news.ycombinator.com