Hacker News Reader

Comment by BozeWolf | Hacker News Reader

I wonder if Apple’s trick would work for python packages. Pay a few (5? 10? 20?) bucks to become a Pypi developer/sponsor, it would also pay pypi’s operational bills. It raises the bar for malicious actors.

Additionally, if pypi provides keys to developers, pypi can also revoke certificates for developers making malicious packages.

It would need a system which checks package signatures on startup of a python app, or maybe there is some other way to do that. Pip —check or some thing which then runs in pipelines, specifically meant to check for malicious packages each day.

To decrease the barrier of entry on pypi, students could identify with their student number. Or pypi could work with a system where you have trusted users and packages and untrusted users. A bit like the blue checkmark, but without the negative connotation.

View on HN

That seems like a way to kill pypi as a popular service. For the 3 or so packages I have, I'd probably just change the description to pull from another location rather than pypi. It's not that I can't afford it, it's just that I don't want to end up paying for another subscription if rules change.

There are also people who wouldn't be able to pay for legal reasons. It would also stop teenagers who don't need the hassle of getting parents to pay.

I think you misinterpreted the comment. The Payment would be required for people wanting to PUSH to pypi. Developers and people PULLING from pypi would remain free. The problem is garbage and malicious submissions.

I don't think the comment you're replying to misunderstood the comment. I think the author was saying that they would update the readme of their projects (which they _push_ to a registry, and for which they do not want to pay) to instruct people to pull the package from somewhere else.

You can do that, but it certainly doesn’t help you with project discovery. If you want only distribute via GitHub that’s fine. But as a developer I’d need a compelling reason to use such a project vs one distributed via pypi

That's ok. Many projects don't care about discovery. (As in being popular/brand name, rather than being possible to find if someone needs that specific thing) And even those that do can rely on people searching for "python package for frobnicating" and finding it somewhere else.

I'm publishing my code for the benefit of others, not mine, so if someone writes the same thing and pays to get it published, that's fine with me.

> I’d need a compelling reason to use such a project vs one distributed via pypi

Apart from trivial projects, there's not that many alternatives you can swap out. Your choices may be "use such project or write your own".

Tbh, changing PyPi urls to git urls at GitHub would almost be a win-win. At least GitHub has namespaces and many other indicators of trust, such as issue tracker, stars, git log and so on. You would loose semver however.

Usually checking the official GitHub projects readme for exact name of pip install command is what I normally do anyway. I would never ever use PyPi as a discovery mechanism, it has too many typosquatters and other lookalikes.

If they went with "let's make the default lookup work well with versions in other registries", I'd love that solution.

not to mention pip search doesn't work anymore...

Imagine having to pay to give open source software away for free. Sounds like a great way to ensure the death of PyPI.

I already host my own packages on my own domain. It was not hard to make a PEP 503 "simple" index.

  python -m pip install chemfp -i https://chemfp.com/packages/

This is commercial/proprietary software for cheminformatics fingerprints.

I don't feel comfortable hosting it on PyPI as I suspect in the future they will want to pay to host non-FOSS licensed software. Which is definitely their prerogative! By hosting it myself already I end up with less lock-in.

Having done it both way, hosting the packages myself means I also like having full access to the download logs, the ability to replace files when I made a mistake, and I can rsync using my usual SSH keys instead of using PyPI's API token system to upload wheels.

I have a very old version of my software on PyPI, distributed under a free software license. The major downside is that people sometimes download that instead of the most recent version. But since it doesn't work on Python 3.x, they either search and find my web site, with the download instructions, or email me.

I figure I also have keep something on PyPI, to prevent name squatting.

How about assigning a trust level to each package. Level 0 - Free, unchecked package, Level 1 - reviewed by at least one moderator, Level 2 - developer bought a $100 license + reviewed, etc. Then it's up to package consumer to decide which package they trust.

FD: I’ve done some work on PyPI, but I am not an admin and everything below is an independent opinion/understanding.

PyPI’s operational costs are, to my understanding, mostly covered: hosting is graciously provided by a sponsor, and the PSF currently funds roles for its develop, administration, and security. More funding is always good and I believe the PyPI admins are looking to enable payments through the newly released “Organizations” feature[1].

Edit: and to make it more clear: payments for Organizations would be principally aimed at corporations and other groups.

> Additionally, if pypi provides keys to developers, pypi can also revoke certificates for developers making malicious packages.

There are currently plans in progress to allow PyPI users to upload Sigstore[2] signatures for packages.

That won’t directly address the spam issue, however — signatures will be opt-in (by necessity, due to the size of the packaging ecosystem), and no codesigning scheme can prevent a spammer from simply assuming a new identity (especially when new identities are “free,” as they normally are.)

Separately, revocation itself is a nasty problem for packaging ecosystems to deal with: ecosystems with trillions of dollars of value behind them (like the Web PKI) struggle with it, so it’s not immediately clear that it would be anything other than an additional operational burden.

Similarly for reputational systems: they’re difficult to operationalize without additional maintainer burden. That’s not to say that they’re necessarily bad or impractical for PyPI’s purposes; only that I’m not aware of a successful use of them in an open source packaging ecosystem. Compare, for example, PGP’s WoT failures.

[1]: https://blog.pypi.org/posts/2023-04-23-introducing-pypi-orga...

[2]: https://www.sigstore.dev/

Web of Trust failed because it lacked motivating reasons to use it, and had a very weird idea about how trust is represented (i.e. the idea of partial trust keys meaning... What exactly?)

What was missing was a role/application based idea of trust chains. i.e. what I trust to do my banking is quite different to what I trust for software reputation.

Because ultimately we're operating a web of trust system now on the internet, it's just informally specified.

> Web of Trust failed because it lacked motivating reasons to use it, and had a very weird idea about how trust is represented (i.e. the idea of partial trust keys meaning... What exactly?)

This is true, but it's also just one among several reasons that the PGP WoT failed. Some are purely implementation and design failures that wouldn't imply to a better scheme than PGP, but others are still intractable in the general case (like timely revocations and "strong set" maintenance).

> Because ultimately we're operating a web of trust system now on the internet, it's just informally specified.

Could you elaborate on what you mean by this? The trust scheme that the modern web is built on is a well-specified hierarchical PKI, i.e. the exact opposite of a web of trust. But it's possible I'm misunderstanding what you mean.

I don't know that I agree with this framing. Pgp could support multiple keyrings, for one.

Bigger, though, is that the internet isn't a web of trust in the same vein? Take out the CA chain, and websites fail trust pretty hard. Even adding them, we wind up accepting certificate pinning on browsers, as nobody has a better option, yet.

And that is building trust to web sites. The stick houses we have to trust communication is basically, "you always call us."

Edit: forgot to say that partial trust is pretty close to "believed, but I wouldn't bank with this identity."

Didn’t know sigstore. The idea looks promising. They should work on marketing though. “Their” technology would work on any package repository, right? Also for npm and the likes.

Not only would it work for NPM, but it already does[1] :-)

[1]: https://blog.sigstore.dev/npm-public-beta/

I maintain a few niche (electric power systems) packages, and I wouldn't mind a one-time or yearly fee, or a fee per project created. I say this as a Brazilian who lived in the middle of nowhere and managed to have a website in the 90's as a teen. If a monetary fee is not desirable, some other hurdle/challenge would probably work fine.

Recently I've seen someone on Reddit trying to automate the creation of PyPI projects through GitHub Actions. The person was complaining that the first deployment couldn't use an API key for that project since it didn't exist. So I'm not surprised some people are trying to do the same for malicious purposes.

The PyPI front page lists 455k projects. If you search for "test", you'll see there's a lot of throwaway projects (note that test.pypi.org is a thing). I'm mostly an EE researcher and I'm not sure students need a low barrier to entry to PyPI, since pip and other tools support installing from GitHub without too much hassle and there are also other non-PyPI package indices. Student packages/projects tend to be abandoned soon after graduation. An archived repo (with a license...), on GitHub or somewhere else, sounds more reasonable and also has more visibility that could end in code reuse someday (through the service's own search and search engines in general). I'd love to understand why so many people repeat this meme that student and teens need trivial access to production infra like PyPI.

So, I'd say being too inclusive, allowing fully unrestricted trivial creation of projects is kinda foolish. There needs to be some extra step, be it a fee, identity confirmation, manual moderation/approval, or something else. I'm sure the PyPA devs/maintainers have ideas.

> Recently I've seen someone on Reddit trying to automate the creation of PyPI projects through GitHub Actions. The person was complaining that the first deployment couldn't use an API key for that project since it didn't exist. So I'm not surprised some people are trying to do the same for malicious purposes.

Sorry for the tangent, but: you can do this now! If you use trusted publishing, you can register a "pending publisher" for a project that doesn't exist yet. When the trusted publisher (like GitHub Actions) is used, it'll create the project[1].

All of this is supported transparently by the official publishing action for GitHub Actions[2].

[1]: https://docs.pypi.org/trusted-publishers/creating-a-project-...

[2]: https://github.com/pypa/gh-action-pypi-publish

Interesting, thanks for the links. By the way, the one I mentioned was in r/learnpython, which is probably not exactly the ideal audience for such a feature.

No problem. And I agree completely -- it'd be really nice if newcomers could copy a template (or even better, have a tool make one for them) that handles all of this behind the scenes.

> There needs to be some extra step, be it a fee, identity confirmation, manual moderation/approval, or something else. I'm sure the PyPA devs/maintainers have ideas.

When I was younger, I always thought computing was so incredibly cool, because me, just some blind kid in Florida, could contribute and make things and share things and just ...participate. I would talk to friends trying to go in to other careers, and excitedly talk about what I was working on and be curious why they never did anything related to what they wanted to do when they grew up.

Now, I understand how this comes about, bit by bit, with the best of intentions.

And I hate it.

Please, just no. If you want to set up a corporate only, super-sekret clubhouse of a PyPi that only the authorized developers can push to, well, the source code for PyPi is right here[0]! And here's Stripe[1]! But please don't break even more of the open, free Internet that I grew up with, I'm pleading with you.

[0]: https://github.com/pypi/warehouse

[1]: https://dashboard.stripe.com/register

The ones who broke the commons are the ones who abused it by uploading malware and lazy student projects. You shouldn't blame the host for trying to maintain standards in light of this. When you have a tragedy of the commons scenario, the solution is to regulate the commons. Otherwise it will lose all value for everybody.

You can always have a trusted community member to waive the payment requirement, so anyone who demostrates genuine effort can have it for free.

This. Also enterprises using specific (still untrusted) packages could pay it for the developer. There must be a way to do this. But in true spirit of python the system should be as open as possible.

Universities could host gitlab instances with pypi registries built-in.

Start asking maintainers for SBOMs (this is rhetorical; please don't) and you'll find out how disinterested they are in doing even more work for others. They understand that there's a problem, but don't have the will to deal with extra work when they're already short on time.

Open source projects tend to be maintained by a tiny number of people who are scratching their own itch, not signing up for more barriers.

As long as devs could be identified properly (ID selfie + ID closeup + small payment with card in same name) and students do the same with a smaller amount, I think that could work. Refund of protection deposit when you close your account. System of honor based on account age. The problem is it's incredibly hard to properly ID people. If fraudsters can get an ID+card they can make a convincing fake selfie ID photo from social media pictures. Plus accounts can just be stolen.

Hahaha no. You physically cannot do it. Read PEP-508. Python can install packages from anywhere. Even if you start with PyPI, the dependency can be specified as hosted on a random unrelated resource.

And, on principle, I wouldn't pay PyPI anything. I want them gone, I don't want my money to feed a bunch of incompetent people who make my life miserable. So, if they were to implement this idea, I'd be hosting my packages on GitHub or Gitlab etc. and have dependencies link to those sources.

You seem pleasant.

Reply on news.ycombinator.com