Lumos: Learning Agents with Unified Data, Modular Design, and Open-Source LLMs

It depends. LLMs are text completion bots. If you want it to chat with you, you prompt it with 'The following is a chat log between user and bot: User: Hi\n\nBot:' and it will keep writing as if it got to the end of an actual chat log and is completing what it thinks could be the next part of it, starting with Bot:. In this case it will take cues from the writing before, so any stylistic or misspelling issues could be imitated.

However, instruct finetunes (like GPT-4) will generally not do this. They are trained on a set of data with 'Instruction' and 'Ideal completion', and are filling the 'ideal completion' part, and thus any errors on the instruction should not be detrimental, as long as it can understand what has been written.

Of course this also depends on the 'unseen' parts of the prompting. An example of this is given above -- you don't write the initial part of the prompt as a user, nor do you add the start and end sequences. If you don't use the type of prompting that a model was trained on, you can get strange results.

A few bits we've learned from building louie over the last year+:

* Training on the internet means fault tolerance out-of-the-box

* Rule-of-thumb: Sub-gpt4 models are derpy, while gpt4/claude are reasonable. The dumber the model, the more it gets tripped up on basics and the more you need to engineer.

* In areas it's already confused, prompt errors can push it over the edge: When asking for JSON, will it output JSON strings, or Python literals?

* Even GPT4 grade will get tripped up on things like negations. So simpler grammar helps. As does emphasizing where to pay attention. REALLY.

* They're simple-minded. So asking GPT4 to do multiple things, especially over a long duration, gets weird. (Long context is a bit of a myth.)

* They're sensitive to persona. Instructing it to answer carefully as a professional would helps improve quality. The opposite is less clear - the base model is just ok, and typos seem to be similarly ok.

* Agents can easily 2-100X quality, so after basics in the prompt, better to focus on the code outside the prompt.

I believe Gemini has a 1M context window.

Guess you are looking for this - https://github.com/allenai/lumos/blob/main/README.md