Hacker News Reader

Comment by alexflint | Hacker News Reader

Yes but how much CPU and memory are you willing to spare? Parsing is quite CPU-intensive, hence the CPU drag you often see when IDEs start indexing, and type inference involves a lot of unpredictable lookups all over the index, so much of it needs to be in memory to get reasonable performance (yes this is still true if we're talking SSD).

To see why: when you type "x.foo()" we need run type inference on the complete data flow chain that produced the value "x", so that we know which particular "foo" you're using. Throughout this analysis we may also need to know a lot about the python libraries you're using, since you may be passing values into and out of arbitrary third party libraries. If each of the steps in this chain triggered an SSD read then you'd often have a multi-second lag between hitting a key and seeing the result.

View on HN

My editor had no problem indexing all of Chromium (fairly large project). It also indexed external libraries. It indexes new code as you write it so type a foo function, next time you type foo you get help immediately. It added standard libraries by default and you can add any other library (like I have it indexing Unity3D's mono libraries).

I didn't notice more than a 100-200ms delay in seeing the result (which happened in other threads so no effect on my editing). About the same I'd expect with a round trip over the the internet. It shows both help at the cursor as well as definitions and references in another pane in that time.

It doesn't look as slick as Kite but it also seems to suggest it's possible to do this all locally. If nothing else you at least have some context (the language) so you don't have to search all data, only data relevant to that language. You even know where in the language I am so you know when to search ids and which subset of ids to search.

On top of that you're basically going to have me sending a gig of source to you to index something like Chromium which will take hours on my crappy connection.

Let me be clear, I think kite looks amazing and I'd be happy to pay for it if it was local. Maybe you download the DB to my machine. I'm not nearly as comfortable with you reading my terminal though. I'm sure you can turn that feature off but that's a feature I liked. Turing features off = less interesting

What editor?

I'm not an OP, but Eclipse does quite well indexing fairly large projects.

This argument seems decent enough, but a couple of things come to mind.

1. Wouldn't multiple HTTP requests be just as slow? (especially if you're on a slow cellular connection, which I am at a coffee shop). Are there any benchmark tests you've done?

2. Is there a reason you couldn't select libraries to download at the start of your project? It's not like you switch libraries too much, especially after initial exploration. I don't think it'd break workflow too much to check some boxes or pick a few packages, I do it already in sublime.

3. To allay concerns about storage for some, couldn't you dump the data after some point? It seems like to be useful you'd only need it while someone is coding, and certainly not long term.

Just some thoughts, would love to get a response. Like others have said, I love the idea, but especially at my work, I don't think I could actually get permission, which puts me in a grey area.

In addition to what the others said, I'll note that PyCharm is already doing these things and there isn't a multi-second lag between hitting a key and seeing the completions. It is also doing type inference. And sure, there's a bit of lag when it needs to do a full index, but that shouldn't be often.

All of it. That's why I run an X99 system with 32GB of 3200MHz RAM at home, and a i7-6700 w/ 16GB 2866MHz RAM in my laptop.

Computational power is so insanely cheap these days it doesn't make any sense to not throw the as much power at your programming environment as you can, IMO.

Those aren't insanely cheap computers - those are top-of-the-line machines - so you can't really design an application to require that kind of hardware to work effectively and expect the user response to be good.

But you're right, that kind of computing power is cheap compared to what you can do with it. If the $1000 in lifetime cost that separates your system from a baseline enables you to be a few percent more productive, it will quickly pay for itself.

That hardly seems necessary. Only so many objects have a foo method to find. With free cores, search from both ends and meet in the middle.

> To see why: when you type "x.foo()" we need run type inference on the complete data flow chain that produced the value "x", so that we know which particular "foo" you're using. Throughout this analysis we may also need to know a lot about the python libraries you're using, since you may be passing values into and out of arbitrary third party libraries.

I don't understand. The libraries I'm using need to be on my machine anyway. If you're doing type inference, there's no need to do the whole process each time: you incrementally add type information and keep it in an index in RAM and the disk. Can't there be a hybrid approach, where your servers are contacted only for libraries that are searched over but not imported yet?

> Yes but how much CPU and memory are you willing to spare?

Hell, I'm using 50% of my CPU to have 60 fps animated 3d fractals as my wallpaper. Surely I can spare some to parsing.

Reply on news.ycombinator.com