ivanovm

working on it!

we use GPT-4-turbo across the board, 3.5 doesn't handle multi-step reasoning nearly well enough to be useful here.

We'll have to figure out how to extract codebase conventions efficiently and inject them into the coding prompts. Using up all 128k tokens has given us worse results (common theme), but hopefully with newer 1m context window models this concern away

that's fair, I get this sentiment with other products I try as well. we have work to do on simplifying the app further, especially on first contact

Ok I got a first cut of Ruby parser in - the latest commit on Rails was a little sluggish but it works!

Right now the parser intentionally tags modules as classes, we made some assumptions in the generic parser code for other languages that don't quite align with ruby's notion of modules, but that'll be adjusted in the near future.

That is the case today. Very hard to beat off-the-shelf models from a standing start, but hoping to get into it in the future

Noted! I'll see if I can get Ruby to work in the next hour or two

heads up, we currently support ts/js, python, and rust pretty well + the common config filetypes like yaml and json

some languages are still experimental - C, C++, Java, Go, Scala, HTML, CSS

there were also some blips on OpenAI availability, they say things are back to normal now though

thanks, pushing a hotfix for this now I've also seen some people running into issues with C/C++ indexing and I'm fixing that too!

Building on the self-driving analogy - our goal is to create a good "GPS" interface where you can set your destination and chart a path you wanna take while driving. In that frame Copilot is like an advanced drive-by-sensors system that sees the road around you and suggests some turns you might wanna take based on the drive so far. What could make this really great is the ability to build up the big picture of the code change so the in-line suggestions are informed by the path ahead.

We'll queue up database schemas, good call.

On RAG, I have a lot of thoughts - it's my pet project to make it work really well. We started out down this road but in practice getting good results out of RAG has been shockingly hard on bigger, more complicated codebases. In many cases when we put this in front of devs that know their code, the RAG part has been a lossy approximation of what they already knew the relevant code to be, so they'd sit there and watch "is this thing going to find the code that I already know I need for this task"

I think the key here will be a robust, language-agnostic symbol graph that maps out high-resolution dependencies, so when you are retrieving `foo`, you can also efficiently retrieve the references to and from of `foo` to some depth (often greater than 1). Some languages have better tools for this kind of symbol resolution than others, but in many cases they're highly bespoke. Github's stack-graphs and Sourcegraph's SCIP are the best attempts I've seen at solving this in a generic way, but neither of them have gotten all the way there yet.

I've been half-seriously thinking about running neovim in headless mode and using it as an interface to all the LSP's for each language, and resolving all the edges of the symbol graph that way - I might still try it one of these days

On design doc convention - I think you're right, it's not completely 1-1 with the term today, and there's not a single definition that everyone agrees on. We're using it as the initial mental model of where we want to plug in, but there might be a better succinct way to describe it

Haven't tried the raw ast yet, curious how that goes

I like Devin's take on rolling up more tools (terminal, editor, browser) in their interface.

We're thinking about this a lot, but our goal is to be very careful about adding complexity to the product (we failed at this in our other endeavors!)

Echoing Rob's sentiment on autonomy, we're more focused on tit-for-tat collaboration between the developer and AI as opposed to "set it and forget it" agent execution. I think both modes will have a role in the dev workflow of the future

This is weird, and it does sound ominous. I went to double check the configuration!

On-prem is on our roadmap, we can definitely make that happen for you. We just went this route for now to make the app easily accessible to everyone