Crate.io: A new kind of Python package index | Hacker News Reader

Crate.io: A new kind of Python package index

crate.io

132 pointsak21747 comments

I am confused as to exactly what Crate.io does differently to PyPi. Yes, it is a better web interface, but what are the actual improvements over PyPi? Why would we use it? Wouldn't it be better for the PyPi website to get a revamp?

As it is now the front page of PyPi https://pypi.python.org/pypi gives us more information than the front page of Crate.

I like that it's quick, PyPi is a bit slow for me at the moment.

I also like the info page for a specific package (https://crate.io/packages/docopt/#description), it's nice and informative, quite easy to read but I do not like that you have to click the info tab to see the classifiers (which python versions it works with, among other things).

What is the key arguments and features that should make us want to use Crate.io over PyPi?

I mostly interface with PyPi via pip on the commandline.

Is crate supposed to be a drop-in replacement? Can we just make pip work against a different base url? Will you launch your own pip replacement?

How is Crate.io better than the current PyPi? The website does not mention any of this, it barely contains any text.

</ramble>, I've just wrote up a few of the questions I had while checking it out, sorry if it's a bit incoherent :-)

Crate is currently an experiment of mine into what a better package system would look like.

It can be used as a drop in replacement against PyPI using the --index-url option of pip. (pip install --index-url=https://simple.crate.io/)

It has a number of security and speed improvements over PyPI some of which I've contributed back to or pushed for PyPI to get overtime.

Would suggest putting up a simple one page description of this somewhere on the site.

> How is Crate.io better than the current PyPi? The website does not mention any of this, it barely contains any text.

But it has a huge ass search field! It's all about the UX, brother; the bigger (and flatter!) the UI, the better the UX, as any UX expert will tell you.

It implements full text search without repeated strcmp() over half a gigabyte of strings, so search completes in reasonable time.

Compare that to PyPi which does exactly what I described, and even during off peak periods, searches can sometimes take up to 20 seconds

Really? Why do they strcmp() over half a gb of strings, that seems pretty stupid and inefficient.

Why not just fork the pypi code and fix it, then?

Have you looked at the PyPI codebase? ;)

The real answer is I have. I'm a PyPI administrator now and have been enacting a lot of these benefits on PyPI itself. Just this weekend I've deployed a CDN in front of all of PyPI so that package downloads have sped up by 1-3 orders of magnitude depending on location.

Previously I had pushed hard for getting a real valid and trusted SSL certificate on PyPI, moving the user provided html/js from packages.python.org to pythonhosted.org in order to prevent CSRF fixation attacks.

Like I said Crate is an experiment. One that has enabled me to push for good changes in PyPI.

I think you're doing good work here, and I agree that PyPI is quite a bit of a mess. One little suggestion/question, though.

CPAN: Comprehensive Perl Archive Network

Rubygems: (self-explanatory)

PyPI: Python Package Index

Crate.io: Crate? Crate.io?

Nothing about the name or any part of the domain really mentions Python, or Python packages or libraries. It's not a bad name, but it doesn't really seem like a name for a large Python package index. What's the exact rationale behind the naming?

Crate is a synonym for Package and I liked the name :)

If you aren't aware, packages in the Rust programming language are called crates.

I believe a krait is a kind of snake if you want to go for that angle.

The only reason I know this is due to playing Elite II.

Ditto :-) There was a Python freighter too!

IIRC that was actually the best (although not the biggest).

If it actually ends up replacing PyPI it could just live on the same domain as PyPi though (provided they are okay with that). But it needs a lot of work on the front end in order to be better :-), I like the backend fixes and speed improvements though.

Ditto. Plane.io.

This is pretty great. A substantial improvement on PyPI. I like the link to ReadTheDocs - something which I often have to hunt for on PyPI

Great commit message: https://github.com/crateio/crate.pypi/commit/94ce379fe386c90...

Working with PyPI tends to make for an angry time.

This looks pretty nice. It would look better if you moved away from the default Bootstrap look. You don't really say what it is, but I'm assuming it's just a front end for information from PyPI.

I don't care about the look, bootstrap whatever.. it works i'm fine.. great work.

Wait until Bootstrap gets in the way…

I'm not the author, but to me this is an attempt to reinvent PyPI from the ground up, with a focus on usability.

If it "looks pretty nice" then that's more than enough -- the author is no doubt trying to solve an actual problem, not just pander to the aesthetic whims of "design" types.

I thought the point was to make PyPI look nicer. I still don't understand what problem they're solving.

Is a mirror of PyPI. Is not only frontend, is a separate service.

Am I nuts or was crate.io a paid filedrop service in the past?

You are thinking of https://letscrate.com/.

How does it pick up the packages? Is there a registration to get on board? I have some PyPi packages that aren't showing up in search, how would I go about adding said packages to this index?

It uses the XMLRPC API to pick up new packages. http://wiki.python.org/moin/PyPIXmlRpc

"Crate is a new kind of Python package index."

Why?

Yes, but what does it do?

Nice. I'd like to sort search results by the number of downloads.

Looks great!

I miss one thing: PGP signature files. I've been checking one of my packages and PyPI is showing the .asc file but crate only shows the sha256 hash.

I would find either PyPi or Crate.io more useful if either of them provided a method to determine which modules are downloaded most often.

http://mail.python.org/pipermail/distutils-sig/2013-May/0208...

Would love to see PyPI or Crate.io list dependencies or provide a way to view them before I download a package...

Unfortunately in the general case that's not information PyPI will know.

Not yet anyways ;)

yeah i like this too. very nice! and would much prefer looking at and using this to pypi.

although i will say i usually just google for a python package and come across a stackexchange that points me to some package, rather than using some index. if you can become some authoritative reference, more power to you.

I get a 500, when I try to sign up using GitHub.

Happens at

    https://crate.io/social-auth/complete/github/?code=123

Nice and clean but I am not sure what problem it is really trying to solve.

Have you tried using the Cheeseshop lately? I have. It was a painfully slow and wretched experience and I ended up browsing multiple pages to find what I was looking for.

What adding about some ability to browse? It is fast!

Maybe I am being dumb but I don't really get it

Pypi.python.org/simple is the heart of everything - do we even use a package repository to browse for packages these days - or do we rely on the blue googles?

Where does the downloads information come from?

Reply on news.ycombinator.com