Thanks for the feedback! That's coming, we've been prioritizing scaling the index and ingest process and haven't had a chance got add that yet. There are a bunch of value-add features like this I am looking forward to knocking out soon.
Because I pushed such a repo yesterday and it’s still not been indexed.
I thought it would be nice to be able to search through them, but I guess the file limit was reached. That’s a shame.
if it works, which I really hope it does, we should have a repo containing every file (sub-25mb) published to pypi.
it's not really useful to clone this, but the git packfile + index seems to be pretty ideal for storing + querying this data, reducing 14tb of compressed packages down to like 100gb or so.
this should enable large scale exploration of the contents of pypi from only your laptop, which is useful if you want to look at how the python language is evolving (how many packages use f-strings over time?).
the rust tool i've built to do the heavy lifting is here: https://github.com/orf/pypi-import-test. i'm learning a lot about git internals, it's quite fun