Show HN: Code Indexer Loop

github.com

28 points·ricklamers··6 comments

This is a project I recently worked on at Definitive. We’re hoping to make it easier to semantically query (changing) source code files. We’d love to see people use it as a building block. If you have any suggestions please leave a comment or open a GitHub issue!

Open articleView on HN

Could anybody show some example queries and answers?

Queries on https://github.com/pypa/flit/tree/main/flit_core/flit_core (omitted tests/)

https://pastebin.com/qyzWm6YC

https://pastebin.com/DDnDKGxd

From the initial look, it looks like you're taking the work of SweepAI, a source-available repo, which gave(?) it to LlamaIndex(1) and then you're just providing a wrapper around the LlamaIndex that provides a way to do an Update statement.

1) https://docs.sweep.dev/blogs/chunking-improvements

Sweep is mentioned as attribution in multiple places a) https://github.com/definitive-io/code-indexer-loop#attributi... b) https://github.com/definitive-io/code-indexer-loop/blob/fd9d...

The difference is packaging it as a consumable PyPI package that can easily be used in a project (they even call out for separating this out into a stand alone project but that they lack the time to do so: https://docs.sweep.dev/blogs/chunking-2m-files#future- )

In addition, we expand and fix the implementation, for example it now supports limiting on token count instead of character count, and we fix some white space inconsistencies in parsing/chunk reconstruction.

Looking forward to diving further in and seeing how you handle white spaces issues as I’ve had massive issues with that already

Please let me know if anything isn’t as expected and I’ll try to look into it

Reply on news.ycombinator.com