Comment by falcolas | Hacker News Reader

Comment by falcolas | Hacker News Reader

Parent Full thread

At first blush, the "we wouldn't be able to ship the entire index along with the client" makes perfect sense. However, I have found that I can store the entirety of the raw Python 2.7 documentation to disk in under 15 MB. It's mostly text, so it compresses absurdly well (2mb, gzipped). And if it was just an index of all the functions with little to no markup, it would be even smaller (this doesn't even account for the fact that all of Python's documentation is typically already on disk, in the modules themselves).

So, given that these indices could be obtained as needed, I'm not sure I buy this argument anymore.

If we can justify the download of a 20mb for a set of plugins, I think that we can justify a 2mb download to document the entirety of a language's standard library.

As a few points of comparison, Atom's download is around 80MB, Docker 110MB, PyCharm is around 175mb, and a ctags file that covers over 8,000 source files (including boto2/3, aws-sdk-go, the python standard library (both 2.7 and 3.5), the go standard library) is about 6 MB compressed.

Right, but the python standard library docs are just a tiny tiny fraction of all the documentation in our index, and the documentation itself is just one corpus out of many.

So, how much megabytes (compressed) does it take to store all of your corpora?

Couldn't it just load based on requirements.txt?

To support your point, Eclipse has zero problem providing auto-completion with JavaDoc information using nothing but local class files. [0] It will also use source or JavaDoc archives to provide full documentation, if they are available.

[0] I know this, because it's a complete pain to use class files which have been stripped of debug data. Suddenly autocompletes look like "someFunction(String arg0, Thing arg1)" with no other supporting information.

EDIT: Just took a look, the indexes for a few workspaces I have are all under 50MB.

Bit more context, a full pypi mirror is 500GB. But I guess that's lots of wheels etc.

Code + tarballs + wheels + history; yeah, I can imagine that 500GB is not that out of the question.

That's a fair comment, but how many libraries does a typical Python projects use? I'm going to go with 100-150,the index needed will be under 100MB.

JavaScript projects tend to have more dependencies, but those tend to be smaller and will have a smaller index.

Solving this 'problem' would lose Kite money, so I don't fault them for not attempting it.

> Solving this 'problem' would lose Kite money, so I don't fault them for not attempting it.

I think it really boils down to: what happens when their servers are compromised; how much liability will Kite assume for the lost IP? My guess is: None.

They don't even have a discoverable privacy policy, just a blog post! Going into the purchase pipeline, there's no service contract, just a "sign up for an account and give us your CC".

Reply on news.ycombinator.com