sudoapps

Awesome! Let me know if you run into any issues with the setup.

Coding agents have changed how I build. Constantly switching between the terminal and an IDE started to feel inefficient, so I wanted a better terminal-first setup where I could manage multiple agent sessions and make quick edits without the overhead of a full IDE. So I built Helm for myself: https://github.com/samirkhoja/helm

Coding agents should help us reduce dependencies overall. I agree Go is already best positioned as a language for this. Using random dependencies for some small feature seems archaic now.

Hot take - Local LLM computing will move to stationary, always on devices (Mac mini & studio). Developers and users will move to lighter, portable devices to interface with their long running agent workers (MacBook Airs & iPads).

Codex built my old startup in a weekend. What took me a few months to build 2+ years ago can now be done with coding agents in 2 days. The last couple months feel like a step change with Codex and Claude.

How do others feel about the pace of innovation in AI? Is there ever a reason to slow down advancements in a given field?

Speed and performance improvements like this are why VSCode has passed Atom and other editors over time.

Very similar to a consensus network. That is probably the future of LLM agents.

This was a quick experiment on creating multiple LLM (GPT) agents with their own objective to see if outputs can be improved.

I can see this being useful as a specialized (fine-tuned) LLM in a chain of LLMs for full autonomy

Autonomous agents could be the next evolution of AI, extending the capabilities of LLMs. Interested to see which agent implementations seem the most promising today.

As this LLM agent architecture continues to evolve and improve, we will probably see a lot of incredible products built on top of it.

This is really interesting. Could this lead to eventually being able to deconstruct these "black-boxes" to remove proprietary data or enforce legal issues?

Completely agree

This wasn't mean't to say that all training would stop. I think, to some extent, the model won't need additional recent data (that is already similar in structure to what it has) to better understand language and interpret the next set of characters. I could be completely wrong, but I still think techniques like transformers, RLHF and of course others will still exist and evolve to eventually get to some higher intelligence level.

OpenAI doesn't let you fine-tune GPT-4 or GPT-3.5 yet (https://platform.openai.com/docs/guides/fine-tuning), but fine-tuning models on a set of documents is still an option but not really scalable if you want to keep feeding it more relevant information over time. I guess it could depend on the base model you are using and its size.

> But then I see model context length getting longer and longer just within the transformer architecture and the training engineering going on.

Do you have any references to this? Seems really interesting if that can be a long term approach.

The article is definitely still high level and mean't to provide enough understanding of what capabilities are today. Some of what you are mentioning goes deeper on how you take these learnings/tools and come up with the any number of solutions to fit the problem you are solving for.

> "Do you use the whole document as context directly? Do you summarize the documents first using the LLM (now the risk of hallucination in this step is added)?"

In my opinion the best approach is to take a large document and break it down into chunks before storing as embeddings and only querying back the relevant passages (chunks).

> "What about that trick where you shrink a whole document of context down to the embedding space of a single token (which is how ChatGPT is remembering the previous conversations)"

Not sure I follow here but seems interesting if possible, do you have any references?

> "What about simply asking the LLM to craft its own search prompt to the DB given the user input, rather than returning articles that semantically match the query the closest? This would also make hybird search (keyword or bm25 + embeddings) more viable in the context of combining it with an LLM"

This is definitely doable but just adds to the overall processing/latency (if that is a concern).

A lot of what prompting has turned into seems silly to me too, but it has shown to be effective (at least with GPT-4).

Agreed, GPT answering based on its own training data has been the best experience by far (aside from hallucinations) and comparing against that is difficult. Embeddings might not even be the long term solution. I think it's still early to really know for certain but models are already getting better at interpreting with less overall training data so there are bound to be some new ideas.

If you are wondering what the latest is on giving LLM's access to large amounts of data, I think this article is a good start. Seems like this is a space where there will be a ton of innovation so interested to learn what else is coming.

For engineers looking to learn how autonomous agents work, this is a high level technical dive into the popular Auto-GPT project.

Great to see them giving attribution and reference links to open source. I don't think co-pilot does this at all.

"To help you code responsibly, CodeWhisperer filters out code suggestions that might be considered biased or unfair, and it’s the only coding companion that can filter or flag code suggestions that may resemble particular open-source training data."