skp1995

> Working on proprietary software means if AI doesn’t run locally, it doesn’t run.

100% agree with you on this.

We want to deploy these AI agents locally and also in a self-hosted fashion for enterprise. The way I imagine this to work is the editor will be a shell and just how LSPs are swapped in and out for different languages or work together, there will be APIs exposed for AI to work with, you want an expert Python AI agent..sure here you go, you want something for Rust.. there we go!

Creating a consistent API layer and getting AI agents to work on top of these APIs is a very important step, and we don't want to or plan on being tied down to just OpenAI.

Enterprise and proprietary software requires local AI agents, and with OSS pushing with great speed here I am sure we will get there in the coming months. (I have prototyped with LLAMA2 locally, on my MacBook Air and its okay, but nowhere close to say GPT4), but the early signs are there and we will be providing this as an option for sure going forward!

weird.. but I do want to reply to your quesiton.

> In this sense it's not just the literature that needs to be reviewed, it's also feature lists (functional at least, and sometimes UI) of prior commercial and open source products.

Truly agree with you on this. Building a whole new editor today would be a hard battle for us, getting adoption even harder. We went with the route of modding VSCode for the sole purpose to ease market adoption and the foundation which VSCode provides. We don't want to stick here, but see how much we can push the editor.

I don't think the editors in the future will even look like the traditional code editors we have today, if AI can write functions, modules, whole services, would it really be code we will look at or something else (I don't want to use the word no-code here... and also because I don't want to be in that world where I am so far away from the code)

Not sure where we will end up in the future with editors, but I do believe its going to look and behave vastly difference. There will be new workflows for us to learn, and discover as we figure out how these AI agents can complement and help us out, an early sign with copilot is how my code has gone from

``` // comment if I want to {code} {mode_code} ```

to

``` // comment {copilot_generated} // comment {copilot_generated} ```

These are just early behaviour changes, but the future does look pretty interesting!

Cursor looks very much in the ballpark and have done a good job. For us, the most important thing is not just code completion or talking with your code (there are many other tools that can do that), but giving the AI agent the freedom to Cmd+Click, Ask for References, all those things that we do in the editor.

Each interaction the user does with the editor needs to be thought out for the AI agent.. I kind of want to tell the AI agent to "follow these code pointers, and figure out how to add a widget on my website" (similar to how you would tell any other engineer in your team).

We want to build a development environment where AI and humans are working together, rather than humans being guided by the AI agent.

We are not there yet, but engineering the editor right now for these AI agents will pay out in the future as the underlying models improve.

Hi I am the cofounder of CodeStory!

In my experience I spend way too much time fixing the output of AI assistants rather than working on the next thing. It slows me down, even after the speed ups.

And generating the right code at first shot is not something I would expect from any engineer (unless we are working with compiled languages), giving access to simple tools like linters and static analysis tools goes a long way, not just for humans but for these AI agents to improve their work.

Hey all! We are building CodeStory, an AI powered mod of VSCode and wanted to share our thoughts and vision on what the future of IDE's will look like. We have some open questions about how we will get there, and would love your opinion and thoughts on this.

So the davinci-text-003 is basically prompt engineering the model to generate our answer. I tried being very specific and used a template which after a bit of debugging I figured out works best which was basically intuition based as you would talk to a human kind of ... https://github.com/theskcd/spchengine/blob/b2bf55268e9245f70...

LexGrep: allows you to search over all Lex Fridman podcasts https://news.ycombinator.com/item?id=34106047

I built LexGrep as a way to see how quickly I can prototype a working ML based project using the APIs which are widely available and hosted online.

- I first started with whisper to generate the transcript but eventually found a dump online where someone did the same thing :V (first step done)

- Next I tried speaker diarisation which is really easy to do in this scenario (as there are mostly 2 speakers at any time) and I could see Lex saying "welcome to my podcast" in almost all of his podcasts which made annotating the transcript easier.

- After this, its the usual create mini embeddings from a single podcast (I used OpenAI's text-embedding-ada-002) to generate the embeddings. I had to model the text in a specific way "This is a transcript between Lex Fridman and John Carmack.." (this adds the context for the conversation and helps powers query like: What did John Carmack think about {blah}) as the embeddings from the podcast with John Carmack will be closer than say some other podcast.

- Pinecone DB provided a free tier for doing cosine similarity of the generated embeddings which worked out perfectly for my use-case and I could generate with a high enough accuracy the related embeddings from the transcript.

- The final piece here is using davinci-text-003 which takes in the transcript and context and after a bit of prompt engineering (more like getting the LLM to give me the right thing and also say "no" when it can't), I have a working end to end search.

Its really exciting how this all comes together and the APIs allowed me (someone who has been a systems engineer for so so long) try out Machine Learning and quickly see it being beneficial and also fun.