nutellalover

Hey, I teach a class at Stanford covering among other things how to effectively use AI across the full SDLC. Hope it's helpful! https://themodernsoftware.dev/

Codeviz is such a cool project. Love having it in VSCode.

Big fans of Cursor ourselves. One of the goals with this library is to make it easy for maintainers of OSS projects to expose chat support functionality to their users in a very streamlined, easy-to-setup fashion.

So yes you can certainly use to index and query your own repos for yourself, but it's also a way to get more of your OSS lib users onboarded.

Happy to help!

At the beginning, we started with qualitative "vibe" checks where we could iterate quickly and the delta in quality was still so significant that we could obviously see what was performing better.

Once we stopped trusting our ability to discern differences, we actually bit the bullet and made a small eval benchmark set (~20 queries across 3 repos of different sizes) and then used that to guide algorithmic development.

Great question. For most small repos (10-20 source files) this works incredibly well out-of-the-box.

We stress-tested with repos like langchain, llamaindex, kubernetes and there the retrieval still needs work to effectively return relevant chunks. This is still an open research question.

The truth is we started there. But for any reasonably-sized, complex codebase this just isn't going to work as the context window isn't sufficient and moreover it becomes harder for the LLM to reason over arbitrary parts of the context.

For the time being, indexing and retrieving a good collection of 10-20 code chunks is more effective/performant in practice.

This is a great read Simon.

Definitely agree that the trend is toward lower cost where a lot of these use-cases are unlocked. Especially as all the major 3rd party LLM providers scramble to ship better models to retain mind-share.

Thanks for the note! We welcome contributions!

We ran some qualitative tests and there was a quality difference. In fact, benchmarks show that trend to generally hold: https://archersama.github.io/coir/

That being said, our goal was to make the library modular so you can easily add support for whatever embeddings you want. Definitely encourage experimenting for your use-case because even in our tests, we found that trends which hold true in research benchmarks don't always translate to custom use-cases.

You can certainly apply to a private repo. If you want to ensure data stays local, you would have to add support for an OSS embedding/LLM model (of which there are many good offerings to pick from).

Yup! We use tree-sitter and parse it at the file-level.

Thanks!

This is a great idea. Definitely something we plan to support.

The starter config is Openai embeddings + llm, pinecone vector store, gradio for the UI. But it's customizable so you can swap out whatever you want easily.

Feel free to submit an issue on the repo and we'll get to it!

Thanks for the request! This is on our roadmap, as is supporting Github issues and eventually external documentation/code discussions from Slack, Jira/Linear, etc.

OP here! I love this stress test. Will index and get back to you!

OP here.

Thanks for taking the time to check out the project and for your very insightful comments. It appears like Amazon's storyteller was discontinued in 2015 so it didn't end up being as useful as promised.

I think the real power is in what you described: "how to translate the story from script to screen for optimum storytelling power and production cost." This is a nuanced question because that translation seems difficult to get right (how does one incorporate signals about production cost, etc.) Would love to get more of your thoughts here.

Does legislature like this actually have a chance at passing? Or is this just bombastic liberal rhetoric to ignite the campaign trail?

I really don’t understand this aversion to trade schools as a viable career path. The college-is-the-only-path-to-success is distinctly American and to me seems more like a narrative perpetuated so that parents feel justified paying 80k a year to send their kids to any back-alley no-name school in the states. I think it’s pretty disgusting and exploitative.

Do they have any data/experiments to support this elegant-sounding theory?

That bee is huge! It seems they only found a single specimen -- doesn't that mean that the species will inevitably become extinct (i.e. as soon as the female dies)?