saqadri

We already support vector stores! If you go to https://lastmileai.dev/models, you can New > "Tune new model", which under the covers creates embeddings for your data (either file upload or scraped from a website). You get a new "Model Fork" from this, which you can then use in a Workbook.

We're working on more complex scenarios with vector stores, including API integrations.

We'd love to see how people use them to understand that better. Our initial hypothesis is that AI Workbooks fit that sweet spot where a chat interface isn't enough, and a Jupyter notebook is too heavyweight. So most likely this will be helpful for technical users and teams who want to experiment/prototype, and collaborate on the results.

Ads/marketing is definitely one vertical, but even for prompt engineering in other verticals we can see this be valuable.

We're especially excited about the multi-modal usecases where you chain multiple models together, but would like the community feedback direct our product direction.

While building this we were pleasantly surprised how intuitive the interface feels for prototyping. For example, you can insert chat cells in-between other chat cells, which affects the overall messages (and response from the model). You can tweak the model parameters at every cell. And once you're happy with it you've got a workbook saved that you can revisit in the future (including text annotations).

I'd love to see any workbooks you create for your debugging scenario. Please share them to see if we can improve that scenario further.

Absolutely! We are working on that right now. Currently you can copy the output of one cell and manually use it as input for another cell, but we will be making that flow much simpler soon.

We agree! Curious what you've seen with Streamlit -- what kind of workflows are people finding most valuable? We are looking for feedback for future product direction.

We are working on that at the moment! We currently have a Python SDK that can be imported into a Jupyter notebook for more advanced scenarios (https://github.com/lastmile-ai/lastmileai-python).

We are working on updating the SDK for more advanced scenarios (e.g. running bulk evaluations, comparing different workbooks programmatically, etc.). Are there any specific workflows you'd like to see enabled in self-hosted Jupyter notebooks?

Hey folks! I'm Sarmad from the LastMile AI team. We'd love your feedback on this as you try it out. Here's a video example of using AI Workbooks: https://www.youtube.com/watch?v=19vRQQNZLFo

Example Workbook from the video: https://lastmileai.dev/workbooks/clj530sqs000znztcmd5qr7v6

Author here. By all means, go ahead!

Author here. It's such early days in this space and things are evolving quickly -- OpenAI's plugins system is also still in Alpha. We think it's a good time to have a conversation and ideally build an open plugin protocol that everyone can benefit from. It will help unlock even more innovation in this space. We have seen this in the evolution of language servers after Language Server Protocol, and how that benefited so many more developers and gave us better IDEs.

Author here. This section of the readme has more information: https://github.com/lastmile-ai/llama-retrieval-plugin#retrie...

It does use a vector database (pinecone, weaviate, etc.) to store embeddings. The embeddings are created using OpenAI's text-embedding-ada-002 model, but that's not a requirement. In fact we are looking at embeddings generation through BERT or RoBERTa to benchmark performance.

At prompt time, the plugin retrieves the nearest embeddings to the prompt, and inserts them into a more complete prompt before sending it to the model.

Author here. We definitely agree these models are really powerful and can figure out a lot on their own. A protocol wouldn't be restrictive to their flexibility, but it'll allow different clients running different models to still be able to interact with the same plugins.

Defining a standard around external memory, authentication, rules-based engines for preprocess/post-processing, and declarative or dynamic chaining of actions can help make plugins model-agnostic, and benefit all of us as developers and users.